Multi-purpose robots and computer program products, and methods for operating the same

ABSTRACT

Robots, methods, and computer program products for training and operating (semi-) autonomous robots to complete multiple different work objectives are described. A robot accesses a library of reusable work primitives from a catalog of libraries of reusable work primitives, each reusable work primitive corresponding to a respective basic sub-task or sub-action that the robot is operative to autonomously perform. A work objective is analyzed to determine a sequence (i.e., a combination and/or permutation) of reusable work primitives that, when executed by the robot, will complete the work objective. The robot executes the sequence of reusable work primitives to complete the work objective. A robot can be deployed with an appropriate stored library (or access to an appropriate library) of reusable work primitives, based on what the robot is expected to do, or what service category or role the robot will operate in.

TECHNICAL FIELD

The present robots, computer program products, and methods generallyrelate to controlling operation of said robots and computer programproducts, and particularly relate to multi-purpose robots that arecapable of at least semi-autonomously completing multiple different workobjectives.

DESCRIPTION OF THE RELATED ART

Robots are machines that may be deployed to perform work. Robots maycome in a variety of different form factors, including humanoid formfactors. Humanoid robots may be operated by tele-operation systemsthrough which the robot is caused to emulate the physical actions of ahuman operator or pilot; however, such tele-operation systems typicallyrequire very elaborate and complicated interfaces comprisingsophisticated sensors and equipment worn by or otherwise directedtowards the pilot, thus requiring that the pilot devote their fullattention to the tele-operation of the robot and limiting the overallaccessibility of the technology.

Robots may be trained or otherwise programmed to operatesemi-autonomously or fully autonomously. Training a robot typicallyinvolves causing the robot to repeatedly perform a physical task in thereal world, which can cause significant wear and tear on the componentsof the robot before the robot can even be deployed to perform usefulwork in the field.

BRIEF SUMMARY

According to a broad aspect, the present disclosure describes a methodof operation of a robot, the robot comprising a robot body, at least oneprocessor carried by the robot body, and at least one non-transitoryprocessor-readable storage medium carried by the robot body andcommunicatively coupled to the at least one processor, the methodcomprising: identifying a first work objective; accessing a firstlibrary of reusable work primitives from a catalog of libraries ofreusable work primitives, the first library of reusable work primitivesincluding fewer reusable work primitives than the catalog of librariesof reusable work primitives; identifying a first workflow to completethe first work objective, the first workflow comprising a first set ofreusable work primitives available in the first library of reusable workprimitives of the catalog of libraries of reusable work primitives; andexecuting the first workflow by executing the first set of reusable workprimitives in the first library of reusable work primitives.

The method may further comprise: identifying a second work objectivedifferent from the first work objective; accessing a second library ofreusable work primitives from the catalog of libraries of reusable workprimitives, the second library of reusable work primitives includingfewer reusable work primitives than the catalog of libraries of reusablework primitives; identifying a second workflow to complete the secondwork objective, the second workflow comprising a second set of reusablework primitives available in the second library of reusable workprimitives of the catalog of libraries of reusable work primitives, thesecond set of reusable work primitives being at least partiallydifferent from the first set of reusable work primitives; and executingthe second workflow by executing the second set of reusable workprimitives in the second library of reusable work primitives. The methodmay further comprise: identifying at least one additional work objectivedifferent from the first work objective and the second work objective;accessing at least one additional library of reusable work primitivesfrom the catalog of libraries of reusable work primitives, eachadditional library of reusable work primitives including fewer reusablework primitives than the catalog of libraries of reusable workprimitives; identifying at least one additional workflow to complete theat least one additional work objective, the at least one additionalworkflow comprising at least one additional set of reusable workprimitives available in the at least one additional library of reusablework primitives, the at least one additional set of reusable workprimitives being at least partially different from the first set ofreusable work primitives and the second set of reusable work primitives;and executing the at least one additional workflow by executing the atleast one additional set of reusable work primitives in the at least oneadditional library of reusable work primitives.

The method may further comprise: identifying a second work objectivedifferent from the first work objective; identifying a second workflowto complete the second work objective, the second workflow comprising asecond set of reusable work primitives available in the first library ofreusable work primitives of the catalog of libraries of reusable workprimitives, the second set of reusable work primitives being at leastpartially different from the first set of reusable work primitives; andexecuting the second workflow by executing the second set of reusablework primitives in the first library of reusable work primitives. Themethod may further comprise: identifying at least one additional workobjective different from the first work objective and the second workobjective; identifying at least one additional workflow to complete theat least one additional work objective, the at least one additionalworkflow comprising at least one additional set of reusable workprimitives available in the first library of reusable work primitives,the at least one additional set of reusable work primitives being atleast partially different from the first set of reusable work primitivesand the second set of reusable work primitives; and executing the atleast one additional workflow by executing the at least one additionalset of reusable work primitives in the first library of reusable workprimitives.

The robot may include a communication interface communicatively coupledto the at least one processor, and identifying the first work objectivemay include identifying the first work objective based on instructionsrelated to the first work objective received via the communicationinterface.

Identifying the first work objective may comprise autonomouslyidentifying, by the at least one processor of the robot, the first workobjective.

The robot may include at least one sensor; the method may furthercomprise capturing, by the at least one sensor, sensor data representingan environment of the robot; and identifying the first work objectivemay comprise identifying, by the at least one processor of the robot,the first work objective based on at least the sensor data.

The robot may include a communication interface communicatively coupledto the at least one processor, and identifying the first work objectivemay comprise identifying, by the at least one processor of the robot,the first work objective based on context data received by thecommunication interface from a device remote from the robot body.

The robot may include a communication interface communicatively coupledto the at least one processor, and identifying the first workflow mayinclude identifying the first workflow based on an indication related tothe first workflow received via the communication interface.

Identifying the first workflow may comprise autonomously identifying, bythe at least one processor of the robot, the first set of reusable workprimitives.

The robot may include at least a first physically actuatable componentcommunicatively coupled to the at least one processor and the catalog oflibraries of reusable work primitives may include reusable workprimitives performable by the first physically actuatable component;identifying the first workflow may include identifying the first set ofreusable work primitives to include at least one reusable work primitiveperformable by the first physically actuatable component; and executingthe first workflow may include executing, by the first physicallyactuatable component, the at least one reusable work primitiveperformable by the first physically actuatable component.

Identifying the first workflow to complete the first work objective mayinclude identifying a first permutation of a first combination ofreusable work primitives from the catalog of libraries of reusable workprimitives.

Executing the first set of reusable work primitives may includeexecuting, by the at least one processor, processor-executableinstructions stored in the non-transitory processor-readable storagemedium to cause the robot to autonomously perform the first set ofreusable work primitives.

The method may further comprise selecting the first library of reusablework primitives from the catalog of libraries of reusable workprimitives based on the first work objective.

The method may further comprise selecting the first library of reusablework primitives from the catalog of libraries of reusable workprimitives based on the first workflow.

Identifying the first work objective may include identifying the firstwork objective based on the first library of reusable work primitives.

Identifying the first workflow may include identifying the firstworkflow based on the first library of reusable work primitives.

Identifying the first work objective may include identifying the firstwork objective based at least in part on a service category.

The method may further comprise selecting the first library of reusablework primitives from the catalog of libraries of reusable workprimitives based on a service category. The robot may include acommunication interface communicatively coupled to the at least oneprocessor; and the method may further comprise identifying, by the atleast one processor, the service category based on an indication of theservice category received via the communication interface. The methodmay further comprise identifying, by the at least one processor, theservice category based on an indication of the service category storedon the at least one non-transitory processor-readable storage medium ofthe robot.

The catalog of libraries of reusable work primitives may include aplurality of libraries of reusable work primitives directed torespective service categories. Each library in the plurality oflibraries may be directed to a respective service category selected froma group of service categories consisting of: medical service; cleaningservice; housekeeping service; warehouse service; fetch service;delivery service; repair service; painting service; and retail service.

The first library of reusable work primitives may be selected from aplurality of libraries of reusable work primitives, each library ofreusable work primitives including a plurality of work primitives whichis at least partially different from respective pluralities of reusablework primitives included in other libraries of reusable work primitives.Each library of reusable work primitives may be defined according to aset of work objectives performable by reusable work primitives includedin the respective library of reusable work primitives. Each library ofreusable work primitives may be defined according to a place of use forthe respective library of reusable work primitives.

The robot may include a communication interface communicatively coupledto the at least one processor; and accessing the first library ofreusable work primitives from the catalog of libraries of reusable workprimitives may comprise: accessing, via the communication interface, thefirst library of reusable work primitives stored on at least one remotenon-transitory processor-readable storage medium of a device remote fromthe robot. Accessing, via the communication interface, the first libraryof reusable work primitives stored on at least one remote non-transitoryprocessor-readable storage medium of a device remote from the robot maycomprise: transferring, via the communication interface, the firstlibrary of reusable work primitives to the at least one non-transitoryprocessor-readable storage medium of the robot.

Accessing the first library of reusable work primitives from the catalogof libraries of reusable work primitives may comprise: accessing thefirst library of reusable work primitives stored on the at least onenon-transitory processor-readable storage medium of the robot.

According to another broad aspect, the present disclosure describes arobot comprising: a robot body; at least one processor carried by therobot body; and at least one non-transitory processor-readable storagemedium carried by the robot body and communicatively coupled to the atleast one processor, the at least one non-transitory processor-readablestorage medium storing processor-executable instructions which, whenexecuted by the at least one processor, cause the robot to: identify afirst work objective; access a first library of reusable work primitivesfrom a catalog of libraries of reusable work primitives, the firstlibrary of reusable work primitives including fewer reusable workprimitives than the catalog of libraries of reusable work primitives;identify a first workflow to complete the first work objective, thefirst workflow comprising a first set of reusable work primitivesavailable in the first library of reusable work primitives of thecatalog of libraries of reusable work primitives; and execute the firstworkflow by executing the first set of reusable work primitives in thefirst library of reusable work primitives.

The processor-executable instructions may further cause the robot to:identify a second work objective different from the first workobjective; access a second library of reusable work primitives from thecatalog of libraries of reusable work primitives, the second library ofreusable work primitives including fewer reusable work primitives thanthe catalog of libraries of reusable work primitives; identify a secondworkflow to complete the second work objective, the second workflowcomprising a second set of reusable work primitives available in thesecond library of reusable work primitives of the catalog of librariesof reusable work primitives, the second set of reusable work primitivesbeing at least partially different from the first set of reusable workprimitives; and execute the second workflow by executing the second setof reusable work primitives in the second library of reusable workprimitives. The processor-executable instructions may further cause therobot to: identify at least one additional work objective different fromthe first work objective and the second work objective; access at leastone additional library of reusable work primitives from the catalog oflibraries of reusable work primitives, each additional library ofreusable work primitives including fewer reusable work primitives thanthe catalog of libraries of reusable work primitives; identify at leastone additional workflow to complete the at least one additional workobjective, the at least one additional workflow comprising at least oneadditional set of reusable work primitives available in the at least oneadditional library of reusable work primitives, the at least oneadditional set of reusable work primitives being at least partiallydifferent from the first set of reusable work primitives and the secondset of reusable work primitives; and execute the at least one additionalworkflow by executing the at least one additional set of reusable workprimitives in the at least one additional library of reusable workprimitives.

The processor-executable instructions may further cause the robot to:identify a second work objective different from the first workobjective; identify a second workflow to complete the second workobjective, the second workflow comprising a second set of reusable workprimitives available in the first library of reusable work primitives ofthe catalog of libraries of reusable work primitives, the second set ofreusable work primitives being at least partially different from thefirst set of reusable work primitives; and execute the second workflowby executing the second set of reusable work primitives in the firstlibrary of reusable work primitives. The processor-executableinstructions may further cause the robot to: identify at least oneadditional work objective different from the first work objective andthe second work objective; identify at least one additional workflow tocomplete the at least one additional work objective, the at least oneadditional workflow comprising at least one additional set of reusablework primitives available in the first library of reusable workprimitives, the at least one additional set of reusable work primitivesbeing at least partially different from the first set of reusable workprimitives and the second set of reusable work primitives; and executethe at least one additional workflow by executing the at least oneadditional set of reusable work primitives in the first library ofreusable work primitives.

The robot may further comprise a communication interface communicativelycoupled to the at least one processor, and the processor-executableinstructions which cause the robot to identify the first work objectivemay cause the robot to: identify the first work objective based oninstructions related to the first work objective received via thecommunication interface.

The processor-executable instructions which cause the robot to identifythe first work objective may cause the robot to: autonomously identify,by the at least one processor of the robot, the first work objective.

The robot may further comprise at least one sensor; theprocessor-executable instructions may further cause the robot tocapture, by the at least one sensor, sensor data representing anenvironment of the robot; and the processor-executable instructionswhich cause the robot to identify the first work objective may cause therobot to: identify, by the at least one processor of the robot, thefirst work objective based on at least the sensor data.

The robot may further comprise a communication interface communicativelycoupled to the at least one processor; and the processor-executableinstructions which cause the robot to identify the first work objectivemay cause the robot to: identify, by the at least one processor of therobot, the first work objective based on context data received by thecommunication interface from a device remote from the robot body.

The robot may further comprise a communication interface communicativelycoupled to the at least one processor and the processor-executableinstructions which cause the robot to identify the first workflow maycause the robot to: identify the first workflow based on an indicationrelated to the first workflow received via the communication interface.

The processor-executable instructions which cause the robot to identifythe first workflow may cause the robot to: autonomously identify, by theat least one processor of the robot, the first set of reusable workprimitives.

The robot may further comprise at least a first physically actuatablecomponent communicatively coupled to the at least one processor; thecatalog of libraries of reusable work primitives may include reusablework primitives performable by the first physically actuatablecomponent; the processor-executable instructions which cause the robotto identify the first workflow may cause the robot to: identify thefirst set of reusable work primitives to include at least one reusablework primitive performable by the first physically actuatable component;and the processor-executable instructions which cause the robot toexecute the first workflow may cause the robot to: execute, by the firstphysically actuatable component, the at least one reusable workprimitive performable by the first physically actuatable component.

The processor-executable instructions which cause the robot to identifythe first workflow to complete the first work objective may cause therobot to: identify a first permutation of a first combination ofreusable work primitives from the catalog of libraries of reusable workprimitives.

The processor-executable instructions may further cause the robot toselect the first library of reusable work primitives from the catalog oflibraries of reusable work primitives based on the first work objective.The processor-executable instructions may further cause the robot toselect the first library of reusable work primitives from the catalog oflibraries of reusable work primitives based on the first workflow.

The processor-executable instructions which cause the robot to identifythe first work objective may cause the robot to: identify the first workobjective based on the first library of reusable work primitives.

The processor-executable instructions which cause the robot to identifythe first workflow may cause the robot to: identify the first workflowbased on the first library of reusable work primitives.

The processor-executable instructions which cause the robot to identifythe first work objective may cause the robot to: identify the first workobjective based at least in part on a service category.

The processor-executable instructions may further cause the robot toselect the first library of reusable work primitives from the catalog oflibraries of reusable work primitives based on a service category. Therobot may further comprise a communication interface communicativelycoupled to the at least one processor, and the processor-executableinstructions may further cause the robot to identify, by the at leastone processor, the service category based on an indication of theservice category received via the communication interface. Theprocessor-executable instructions may further cause the robot toidentify, by the at least one processor, the service category based onan indication of the service category stored on the at least onenon-transitory processor-readable storage medium of the robot.

The catalog of libraries of reusable work primitives may include aplurality of libraries of reusable work primitives directed torespective service categories. Each library in the plurality oflibraries may be directed to a respective service category selected froma group of service categories consisting of: medical service; cleaningservice; housekeeping service; warehouse service; fetch service;delivery service; repair service; painting service; and retail service.

The first library of reusable work primitives may be selected from aplurality of libraries of reusable work primitives, each library ofreusable work primitives including a plurality of work primitives whichis at least partially different from respective pluralities of reusablework primitives included in other libraries of reusable work primitives.Each library of reusable work primitives may be defined according to aset of work objectives performable by reusable work primitives includedin the respective library of reusable work primitives. Each library ofreusable work primitives may be defined according to a place of use forthe respective library of reusable work primitives.

The robot may further comprise a communication interface communicativelycoupled to the at least one processor, and the processor-executableinstructions which cause the robot to access the first library ofreusable work primitives from the catalog of libraries of reusable workprimitives may cause the robot to: access, via the communicationinterface, the first library of reusable work primitives stored on atleast one remote non-transitory processor-readable storage medium of adevice remote from the robot. The processor-executable instructionswhich cause the robot to access, via the communication interface, thefirst library of reusable work primitives stored on at least one remotenon-transitory processor-readable storage medium of a device remote fromthe robot may cause the robot to: transfer, via the communicationinterface, the first library of reusable work primitives to the at leastone non-transitory processor-readable storage medium of the robot.

The processor-executable instructions which cause the robot to accessthe first library of reusable work primitives from the catalog oflibraries of reusable work primitives may cause the robot to: access thefirst library of reusable work primitives stored on the at least onenon-transitory processor-readable storage medium of the robot.

According to yet another broad aspect, the present disclosure describesa computer program product comprising processor-executable instructionsor data that, when the computer program product is stored in anon-transitory processor-readable storage medium of a robot, and thecomputer program product is executed by at least one processor of therobot, the at least one processor communicatively coupled to thenon-transitory processor-readable storage medium, causes the robot to:identify a first work objective; access a first library of reusable workprimitives from a catalog of libraries of reusable work primitives, thefirst library of reusable work primitives including fewer reusable workprimitives than the catalog of libraries of reusable work primitives;identify a first workflow to complete the first work objective, thefirst workflow comprising a first set of reusable work primitivesavailable in the first library of reusable work primitives of thecatalog of libraries of reusable work primitives; and execute the firstworkflow by executing the first set of reusable work primitives in thefirst library of reusable work primitives.

The processor-executable instructions or data of the computer programproduct may further cause the robot to: identify a second work objectivedifferent from the first work objective; access a second library ofreusable work primitives from the catalog of libraries of reusable workprimitives, the second library of reusable work primitives includingfewer reusable work primitives than the catalog of libraries of reusablework primitives; identify a second workflow to complete the second workobjective, the second workflow comprising a second set of reusable workprimitives available in the second library of reusable work primitivesof the catalog of libraries of reusable work primitives, the second setof reusable work primitives being at least partially different from thefirst set of reusable work primitives; and execute the second workflowby executing the second set of reusable work primitives in the secondlibrary of reusable work primitives. The processor-executableinstructions or data of the computer program product may further causethe robot to: identify at least one additional work objective differentfrom the first work objective and the second work objective; access atleast one additional library of reusable work primitives from thecatalog of libraries of reusable work primitives, each additionallibrary of reusable work primitives including fewer reusable workprimitives than the catalog of libraries of reusable work primitives;identify at least one additional workflow to complete the at least oneadditional work objective, the at least one additional workflowcomprising at least one additional set of reusable work primitivesavailable in the at least one additional library of reusable workprimitives, the at least one additional set of reusable work primitivesbeing at least partially different from the first set of reusable workprimitives and the second set of reusable work primitives; and executethe at least one additional workflow by executing the at least oneadditional set of reusable work primitives in the at least oneadditional library of reusable work primitives.

The processor-executable instructions or data of the computer programproduct may further cause the robot to: identify a second work objectivedifferent from the first work objective; identify a second workflow tocomplete the second work objective, the second workflow comprising asecond set of reusable work primitives available in the first library ofreusable work primitives of the catalog of libraries of reusable workprimitives, the second set of reusable work primitives being at leastpartially different from the first set of reusable work primitives; andexecute the second workflow by executing the second set of reusable workprimitives in the first library of reusable work primitives. Theprocessor-executable instructions or data of the computer programproduct may further cause the robot to: identify at least one additionalwork objective different from the first work objective and the secondwork objective; identify at least one additional workflow to completethe at least one additional work objective, the at least one additionalworkflow comprising at least one additional set of reusable workprimitives available in the first library of reusable work primitives,the at least one additional set of reusable work primitives being atleast partially different from the first set of reusable work primitivesand the second set of reusable work primitives; and execute the at leastone additional workflow by executing the at least one additional set ofreusable work primitives in the first library of reusable workprimitives.

The robot may further include a communication interface communicativelycoupled to the at least one processor, and the processor-executableinstructions or data of the computer program product which cause therobot to identify the first work objective may cause the robot to:identify the first work objective based on instructions related to thefirst work objective received via the communication interface.

The processor-executable instructions or data of the computer programproduct which cause the robot to identify the first work objective maycause the robot to: autonomously identify, by the at least one processorof the robot, the first work objective.

The robot may include at least one sensor; the processor-executableinstructions or data of the computer program product may further causethe robot to capture, by the at least one sensor, sensor datarepresenting an environment of the robot; and the processor-executableinstructions or data of the computer program product which cause therobot to identify the first work objective may cause the robot to:identify, by the at least one processor of the robot, the first workobjective based on at least the sensor data.

The robot may include a communication interface communicatively coupledto the at least one processor, and the processor-executable instructionsor data of the computer program product which cause the robot toidentify the first work objective may cause the robot to: identify, bythe at least one processor of the robot, the first work objective basedon context data received by the communication interface from a deviceremote from the robot body.

The robot may include a communication interface communicatively coupledto the at least one processor, and the processor-executable instructionsor data of the computer program product which cause the robot toidentify the first workflow may cause the robot to: identify the firstworkflow based on an indication related to the first workflow receivedvia the communication interface.

The processor-executable instructions or data of the computer programproduct which cause the robot to identify the first workflow may causethe robot to: autonomously identify, by the at least one processor ofthe robot, the first set of reusable work primitives.

The robot may further include at least a first physically actuatablecomponent communicatively coupled to the at least one processor, thecatalog of libraries of reusable work primitives may include reusablework primitives performable by the first physically actuatablecomponent; the processor-executable instructions or data of the computerprogram product which cause the robot to identify the first workflow maycause the robot to: identify the first set of reusable work primitivesto include at least one reusable work primitive performable by the firstphysically actuatable component; and the processor-executableinstructions or data of the computer program product which cause therobot to execute the first workflow may cause the robot to: execute, bythe first physically actuatable component, the at least one reusablework primitive performable by the first physically actuatable component.

The processor-executable instructions or data of the computer programproduct which cause the robot to identify the first workflow to completethe first work objective may cause the robot to: identify a firstpermutation of a first combination of reusable work primitives from thecatalog of libraries of reusable work primitives.

The processor-executable instructions or data of the computer programproduct may further cause the robot to select the first library ofreusable work primitives from the catalog of libraries of reusable workprimitives based on the first work objective.

The processor-executable instructions or data of the computer programproduct may further cause the robot to select the first library ofreusable work primitives from the catalog of libraries of reusable workprimitives based on the first workflow.

The processor-executable instructions or data of the computer programproduct which cause the robot to identify the first work objective maycause the robot to: identify the first work objective based on the firstlibrary of reusable work primitives.

The processor-executable instructions or data of the computer programproduct which cause the robot to identify the first workflow may causethe robot to: identify the first workflow based on the first library ofreusable work primitives.

The processor-executable instructions or data of the computer programproduct which cause the robot to identify the first work objective maycause the robot to: identify the first work objective based at least inpart on a service category.

The processor-executable instructions or data of the computer programproduct may further cause the robot to select the first library ofreusable work primitives from the catalog of libraries of reusable workprimitives based on a service category. The robot may further include acommunication interface communicatively coupled to the at least oneprocessor, and the processor-executable instructions or data of thecomputer program product may further cause the robot to identify, by theat least one processor, the service category based on an indication ofthe service category received via the communication interface. Theprocessor-executable instructions or data of the computer programproduct may further cause the robot to identify, by the at least oneprocessor, the service category based on an indication of the servicecategory stored on the at least one non-transitory processor-readablestorage medium of the robot.

The catalog of libraries of reusable work primitives may include aplurality of libraries of reusable work primitives directed torespective service categories. Each library in the plurality oflibraries may be directed to a respective service category selected froma group of service categories consisting of: medical service; cleaningservice; housekeeping service; warehouse service; fetch service;delivery service; repair service; painting service; and retail service.

The first library of reusable work primitives may be selected from aplurality of libraries of reusable work primitives, each library ofreusable work primitives including a plurality of work primitives whichis at least partially different from respective pluralities of reusablework primitives included in other libraries of reusable work primitives.Each library of reusable work primitives may be defined according to aset of work objectives performable by reusable work primitives includedin the respective library of reusable work primitives. Each library ofreusable work primitives may be defined according to a place of use forthe respective library of reusable work primitives.

The robot may further include a communication interface communicativelycoupled to the at least one processor, and the processor-executableinstructions or data of the computer program product which cause therobot to access the first library of reusable work primitives from thecatalog of libraries of reusable work primitives may cause the robot to:access, via the communication interface, the first library of reusablework primitives stored on at least one remote non-transitoryprocessor-readable storage medium of a device remote from the robot. Theprocessor-executable instructions or data of the computer programproduct which cause the robot to access, via the communicationinterface, the first library of reusable work primitives stored on atleast one remote non-transitory processor-readable storage medium of adevice remote from the robot may cause the robot to: transfer, via thecommunication interface, the first library of reusable work primitivesto the at least one non-transitory processor-readable storage medium ofthe robot.

The processor-executable instructions or data of the computer programproduct which cause the robot to access the first library of reusablework primitives from the catalog of libraries of reusable workprimitives may cause the robot to: access the first library of reusablework primitives stored on the at least one non-transitoryprocessor-readable storage medium of the robot.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

The various elements and acts depicted in the drawings are provided forillustrative purposes to support the detailed description. Unless thespecific context requires otherwise, the sizes, shapes, and relativepositions of the illustrated elements and acts are not necessarily shownto scale and are not necessarily intended to convey any information orlimitation. In general, identical reference numbers are used to identifysimilar elements or acts.

FIG. 1 is an illustrative diagram of an exemplary robot systemcomprising various features and components described throughout thepresent robots, methods, and computer program products.

FIG. 2 is a flow diagram showing an exemplary method of operation of arobot in accordance with the present robots, methods, and computerprogram products.

FIG. 3 is a flow diagram showing another exemplary method of operationof a robot in accordance with the present robots, methods, and computerprogram products.

FIG. 4 is a flow diagram showing yet another exemplary method ofoperation of a robot in accordance with the present robots, methods, andcomputer program products.

DETAILED DESCRIPTION

The following description sets forth specific details in order toillustrate and provide an understanding of the various implementationsand embodiments of the present robots, computer program products, andmethods. A person of skill in the art will appreciate that some of thespecific details described herein may be omitted or modified inalternative implementations and embodiments, and that the variousimplementations and embodiments described herein may be combined witheach other and/or with other methods, components, materials, etc. inorder to produce further implementations and embodiments.

In some instances, well-known structures and/or processes associatedwith computer systems and data processing have not been shown orprovided in detail in order to avoid unnecessarily complicating orobscuring the descriptions of the implementations and embodiments.

Unless the specific context requires otherwise, throughout thisspecification and the appended claims the term “comprise” and variationsthereof, such as “comprises” and “comprising,” are used in an open,inclusive sense to mean “including, but not limited to.”

Unless the specific context requires otherwise, throughout thisspecification and the appended claims the singular forms “a,” “an,” and“the” include plural referents. For example, reference to “anembodiment” and “the embodiment” include “embodiments” and “theembodiments,” respectively, and reference to “an implementation” and“the implementation” include “implementations” and “theimplementations,” respectively. Similarly, the term “or” is generallyemployed in its broadest sense to mean “and/or” unless the specificcontext clearly dictates otherwise.

The headings and Abstract of the Disclosure are provided for convenienceonly and are not intended, and should not be construed, to interpret thescope or meaning of the present robots, computer program products, andmethods.

A general-purpose robot is able to complete multiple different workobjectives. As used throughout this specification and the appendedclaims, the term “work objective” refers to a particular task, job,assignment, or application that has a specified goal and a determinableoutcome, often (though not necessarily) in the furtherance of someeconomically valuable work. Work objectives exist in many aspects ofbusiness, research and development, commercial endeavors, and personalactivities. Exemplary work objectives include, without limitation:cleaning a location (e.g., a bathroom) or an object (e.g., a bathroommirror), preparing a meal, loading/unloading a storage container (e.g.,a truck), taking inventory, collecting one or more sample(s), making oneor more measurement(s), building or assembling an object, destroying ordisassembling an object, delivering an item, harvesting objects and/ordata, and so on. The various implementations described herein providerobots, computer program products, and methods for initializing,configuring, training, operating, and/or deploying a robot to at leastsemi-autonomously complete multiple different work objectives.

In accordance with the present robots, computer program products, andmethods, a work objective is deconstructed or broken down into a“workflow” comprising a set of “work primitives”, where successfulcompletion of the work objective involves performing each work primitivein the workflow. Depending on the specific implementation, completion ofa work objective may be achieved by (i.e., a workflow may comprise): i)performing a corresponding set of work primitives sequentially or inseries; ii) performing a corresponding set of work primitives inparallel; or iii) performing a corresponding set of work primitives inany combination of in series and in parallel (e.g., sequentially withoverlap) as suits the work objective and/or the robot performing thework objective. Thus, in some implementations work primitives may beconstrued as lower-level activities, steps, or sub-tasks that areperformed or executed as a workflow in order to complete a higher-levelwork objective.

Advantageously, and in accordance with the present robots, computerprogram products, and methods, a catalog of “reusable” work primitivesmay be defined. A work primitive is reusable if it may be genericallyinvoked, performed, employed, or applied in the completion of multipledifferent work objectives. For example, a reusable work primitive is onethat is common to the respective workflows of multiple different workobjectives. In some implementations, a reusable work primitive mayinclude at least one variable that is defined upon or prior toinvocation of the work primitive. For example, “pick up *object*” may bea reusable work primitive where the process of “picking up” may begenerically performed at least semi-autonomously in furtherance ofmultiple different work objectives and the *object* to be picked up maybe defined based on the specific work objective being pursued.

As stated previously, the various implementations described hereinprovide robots, computer program products, and methods where a robot isenabled to at least semi-autonomously complete multiple different workobjectives. Unless the specific context requires otherwise, the term“autonomously” is used throughout this specification and the appendedclaims to mean “without control by another party” and the term“semi-autonomously” is used to mean “at least partially autonomously.”In other words, throughout this specification and the appended claims,the term “semi-autonomously” means “with limited control by anotherparty” unless the specific context requires otherwise. An example of asemi-autonomous robot is one that can independently and/or automaticallyexecute and control some of its own low-level functions, such as itsmobility and gripping functions, but relies on some external control forhigh-level instructions such as what to do and/or how to do it.

In accordance with the present robots, computer program products, andmethods, a catalog of reusable work primitives may be defined,identified, developed, or constructed such that any given work objectiveacross multiple different work objectives may be completed by executinga corresponding workflow comprising a particular combination and/orpermutation of reusable work primitives selected from the catalog ofreusable work primitives. Once such a catalog of reusable workprimitives has been established, one or more robot(s) may be trained toautonomously or automatically perform each individual reusable workprimitive in the catalog of reusable work primitives without necessarilyincluding the context of: i) a particular workflow of which theparticular reusable work primitive being trained is a part, and/or ii)any other reusable work primitive that may, in a particular workflow,precede or succeed the particular reusable work primitive being trained.In this way, a semi-autonomous robot may be operative to autonomously orautomatically perform each individual reusable work primitive in acatalog of reusable work primitives and only require instruction,direction, or guidance from another party (e.g., from an operator, user,or pilot) when it comes to deciding which reusable work primitive(s) toperform and/or in what order. In other words, an operator, user, orpilot may provide a workflow consisting of reusable work primitives to asemi-autonomous robot and the semi-autonomous robot may autonomously orautomatically execute the reusable work primitives according to theworkflow to complete a work objective. For example, a semi-autonomoushumanoid robot may be operative to autonomously look left when directedto look left, autonomously open its right end effector when directed toopen its right end effector, and so on, without relying upon detailedlow-level control of such functions by a third party. Such asemi-autonomous humanoid robot may autonomously complete a workobjective once given instructions regarding a workflow detailing whichreusable work primitives it must perform, and in what order, in order tocomplete the work objective. Furthermore, in accordance with the presentrobots, methods, and computer program products, a robot may operatefully autonomously if it is trained or otherwise configured to analyze awork objective and independently define a corresponding workflow itselfby deconstructing the work objective into a set of reusable workprimitives from a library of reusable work primitives that the robot isoperative to autonomously perform.

Reusable work primitives in the catalog of reusable work primitives canbe organized into a plurality of libraries of reusable work primitives.In this sense, the catalog of reusable work primitives can also bereferred to as a catalog of libraries of reusable work primitives. Eachlibrary of reusable work primitives includes fewer reusable workprimitives than the catalog of work primitives; that is, each libraryrepresents a subset of reusable work primitives of the catalog ofreusable work primitives. For a given deployment, a robot may access (orhave locally stored) a limited amount of libraries of reusable workprimitives, based on the nature of the deployment. This advantageouslysaves storage space and/or processing burden at the robot. Inparticular, for a general-purpose robot, a catalog of work primitivesusable by the robot may include an immense quantity of reusable workprimitives, but in many cases a given deployment, work objective, orworkflow may only require a much smaller number of reusable workprimitives. If the entire catalog were stored and accessed at the robot,this could significantly burden the robot. For example, whenconstructing or executing a workflow, it is computationally intensivefor the robot to navigate through, process, or select appropriatereusable work primitives from the entire catalog of work primitives. Inparticular, a workflow is constructed of a specific combination and/orpermutation of discrete reusable work primitives. Each additionalreusable work primitive considered (i.e. available for inclusion in aworkflow) exponentially increases the total number of possible workflows(because there are exponentially more possible combinations orpermutations of reusable work primitives), and thus also exponentiallyincreases processing burden to identify or construct a workflow. Byinstead accessing one or more limited libraries of reusable workprimitives, this computational burden is significantly decreased. Thisimproves responsiveness and efficiency, and/or reduces complexity andcost of hardware of the robot. Further, the entire catalog of reusablework primitives may occupy a significant amount of storage space at therobot; storing one or more limited libraries of reusable work primitivesinstead of the entire catalog reduces storage burden. Thus, it ispreferable to limit reusable work primitives accessible to the robot tothose likely to be used for a given deployment or service category inwhich the robot operates.

Each library of reusable work primitives is at least partially differentfrom other libraries of reusable work primitives. While libraries mayoverlap and include at least one common reusable work primitive(possibly many more than one), or not, each library includes a pluralityof reusable work primitives which is different by at least one reusablework primitive from other libraries. As an example, a first librarycould include at least one reusable work primitive which is not includedin a second library. The second library could likewise include at leastone reusable work primitive which is not included in the first library.Alternatively, each reusable work primitive in the second library couldbe included in the first library, but the first library includes atleast one additional reusable work primitive which is not included inthe second library (i.e., the first library is a larger set of reusablework primitives which completely overlaps the second library). In suchan example, the first and second libraries are still considered“different” from each other. In some implementations, a first libraryand a second library may be “wholly different” from one another in thatthey each contain a respective unit set of reusable work primitives withno overlap therebetween.

In the context of a robot, reusable work primitives may correspond tobasic low-level functions that the robot is operable to (e.g.,autonomously or automatically) perform and that the robot may call uponor execute in order to achieve something. Examples of reusable workprimitives for a humanoid robot include, without limitation: look up,look down, look left, look right, move right arm, move left arm, closeright end effector, open right end effector, close left end effector,open left end effector, move forward, turn left, turn right, movebackwards, and so on; however, a person of skill in the art willappreciate that: i) the foregoing list of exemplary reusable workprimitives for a humanoid robot is by no means exhaustive; ii) thepresent robots, computer program products, and methods are not limitedin any way to robots having a humanoid form factor; and iii) thecomplete composition of any library of reusable work primitives dependson the design and functions of the specific robot for which the libraryof reusable work primitives is constructed.

A robot may be operative to perform any number of high-level functionsbased at least in part on its hardware and software configurations. Forexample, a robot with legs or wheels may be operative to move, a robotwith a gripper may be operative to pick things up, and a robot with legsand a gripper may be operative to displace objects. The performance ofany such high-level function generally requires the controlled executionof multiple low-level functions. For example, a mobile robot mustexercise control of a number of different lower-level functions in orderto controllably move, including control of mobility actuators (e.g.,driving its legs or wheels) that govern functional parameters likespeed, trajectory, balance, and so on. In accordance with the presentrobots, computer program products, and methods, the high-level functionsthat a robot is operative to perform are deconstructed or broken downinto a set of basic components or constituents, referred to throughoutthis specification and the appended claims as “work primitives”. Unlessthe specific context requires otherwise, work primitives may beconstrued as the building blocks of which higher-level robot functionsare constructed.

In some implementations training a robot to autonomously perform areusable work primitive may be completed in a real-world environment ora simulated environment. Once a robot has been trained to autonomouslyperform a library of reusable work primitives, tele-operation of therobot by a remote pilot may be abstracted to the level of reusable workprimitives; i.e., a remote operator or pilot that controls the robotthrough a tele-operation system may do so by simply instructing therobot which reusable work primitive(s) to perform and, in someimplementations, in what order to perform them, and the robot may havesufficient autonomy or automation (resulting from, for example, thetraining described above) to execute a complete work objective based onsuch limited control instruction from the pilot.

As described previously, “clean a bathroom mirror” is an illustrativeexample of a work objective that can be deconstructed into a set of workprimitives to achieve a goal and for which the outcome is determinable.The goal in this case is a clean bathroom mirror, and an exemplary setof work primitives (or workflow) that completes the work objective is asfollows:

Work Primitive Index Work Primitive 1 Locate cleaning solution 2 Graspthe cleaning solution 3 Locate mirror 4 Aim the cleaning solution at themirror 5 Dispense the cleaning solution onto the mirror 6 Locate thecleaning cloth 7 Grasp the cleaning cloth 8 Pass the cleaning cloth overthe entire surface of the mirror 9 Return to readyA person of skill in the art will appreciate that the exemplary workflowabove, comprising nine work primitives, is used as an illustrativeexample of a workflow that may be deployed to complete the workobjective of cleaning a bathroom mirror; however, in accordance with thepresent robots, computer program products, and methods the precisedefinition and composition of each work primitive and the specificcombination and/or permutation of work primitives selected/executed tocomplete a work objective (i.e., the specific construction of aworkflow) may vary in different implementations. For example, in someimplementations work primitives 3, 4, and 5 above (i.e., locate mirror,aim the cleaning solution at the mirror, and dispense the cleaningsolution onto the mirror) may all be combined into one higher-level workprimitive as “spray cleaning solution on the mirror” whereas in otherimplementations those same work primitives may be broken down intoadditional lower-level work primitives as, for example:

Locate the mirror

Identify the boundaries of the mirror

Aim the cleaning solution at a first location within the boundaries ofthe mirror

Squeeze the cleaning solution

Aim the cleaning solution at a second location within the boundaries ofthe mirror

Squeeze the cleaning solution

Etc.

Based on the above example and description, a person of skill in the artwill appreciate that the granularity of work primitives may vary acrossdifferent implementations of the present robots, computer programproducts, and methods. Furthermore, in accordance with the presentrobots, computer program products, and methods the work primitives areadvantageously “reusable” in the sense that each work primitive may beemployed, invoked, applied, or “reused” in the performance of more thanone overall work objective. For example, while cleaning a bathroommirror may involve the work primitive “grasp the cleaning solution,”other work objectives may also use the “grasp the cleaning solution”work primitive, such as for example “clean the toilet,” “clean thewindow,” and/or “clean the floor.” In some implementations, workprimitives may be abstracted to become more generic. For example, “graspthe cleaning solution” may be abstracted to “grasp the spray bottle” or“grasp the *object1*” where the *object1* variable is defined as“*object1*=spray bottle”, and “locate the mirror” may be abstracted to“locate the object that needs to be sprayed” or simply “locate*object2*” where “*object2*=mirror”. In such cases, the “grasp the spraybottle” work primitive may be used in tasks that do not involvecleaning, such as “paint the wall” (where the spray bottle=spray paint),“style the hair” (where the spray bottle=hairspray), or “prepare thestir-fry meal” (where the spray bottle=cooking oil spray).

FIG. 1 is an illustrative diagram of an exemplary robot system 100comprising various features and components described throughout thepresent robots, computer program products, and devices. Robot system 100comprises a robot body 101 with a first physically actuatable component102 a and a second physically actuatable component 102 b mechanicallycoupled to body 101. In the illustrated implementation, first and secondphysically actuatable components 102 a and 102 b each correspond to arespective robotic hand, though a person of skill in the art willappreciate that in alternative implementations a physically actuatablecomponent may take on other forms (such as an arm or leg, anon-hand-like end effector such as a cutter or suction tube, or anyother form useful to the particular applications the robot is intendedto perform). Robotic hand 102 a emulates a human hand and includesmultiple fingers 121 a, 122 a, 123 a, and 124 a and an opposable thumb125 a. Robotic hand 102 b is similar to a mirror-image of robotic hand102 a while corresponding details are not labeled for robotic hand 102 bto reduce clutter. Robotic hands 102 a and 102 b may be physicallyactuatable by a variety of different means, including electromechanicalactuation, cable-driven actuation, magnetorheological fluid-basedactuation, and/or hydraulic actuation. Some exemplary details ofactuation technology that may be employed to physically actuate robotichands 102 a and 102 b are described in U.S. patent application Ser. No.17/491,577 and U.S. Provisional Patent Application Ser. No. 63/191,732,filed May 21, 2021 and entitled “Systems, Devices, And Methods For AHydraulic Robotic Arm”, both of which are incorporated by referenceherein in their entirety.

Robot body 101 further includes at least one sensor 103 that detectsand/or collects data about the environment and/or objects in theenvironment of robot system 100. In the illustrated implementation,sensor 103 corresponds to a sensor system including a camera, amicrophone, and an inertial measurement unit that itself comprises threeorthogonal accelerometers, a magnetometer, and a compass, though inother implementations any appropriate type of sensor could be included.

For the purposes of illustration, FIG. 1 includes details of certainexemplary components that are carried by or within robot body 101 inaccordance with the present robots, computer program products, anddevices. Such components include at least one processor 130 and at leastone non-transitory processor-readable storage medium, or “memory”, 140communicatively coupled to processor 130. Memory 140 stores at least onelibrary of reusable work primitives 141 (which may or may not include alibrary of reusable grasp primitives for either or both of robotic hands102 a and/or 102 b depending on the implementation) andprocessor-executable instructions 142 that, when executed by processor130, cause robot body 101 (including applicable actuatable componentssuch as either or both of robotics hands 102 a and/or 102 b) toselectively and autonomously perform at least one reusable workprimitives in library of reusable work primitives 141. In someimplementations, memory 140 stores an entire catalog of reusable workprimitives usable by the robot. To reduce processing burden, the catalogof reusable work primitives is organized into a plurality of librariesof work primitives, each library including fewer reusable workprimitives than the catalog of reusable work primitives. The at leastone processor 130 accesses each library as needed to accomplish a givenwork objective, execute a given workflow, or select work primitives fora workflow. Depending on the specific implementation,processor-executable instructions 142 may further includeprocessor-executable instructions (e.g., a computer program product)that cause robot system to perform any or all of methods 200, 300,and/or 400 described herein.

Processor 130 is also communicatively coupled to a wireless transceiver150 via which robot body 101 sends and receives wireless communicationsignals 160 with an exemplary teleoperation system 170. To this end,teleoperation system 170 also includes a wireless transceiver 171.

For the purposes of illustration, teleoperation system 170 includes botha low-level teleoperation interface 180 and a high-level teleoperationinterface 190. Low-level teleoperation interface 180 includes a sensorsystem 181 that detects real physical actions performed by a human pilot182 and a processing system 183 that converts such real physical actionsinto low-level teleoperation instructions that, when executed byprocessor 130, cause robot body 101 (and any applicable actuatablecomponents such as hands 102 a and/or 102 b) to emulate the physicalactions performed by pilot 182. In some implementations, sensor system181 may include many sensory components typically employed in the fieldof virtual reality games, such as haptic gloves, accelerometer-basedsensors worn on the body of pilot 182, and a VR headset that enablespilot 182 to see optical data collected by sensor 103 of robot body 101.High-level teleoperation interface 190 includes a simple GUI displayed,in this exemplary implementation, on a tablet computer. The GUI ofhigh-level teleoperation interface 190 provides a set of buttons eachcorresponding to a respective action performable by robot body 101 (andapplicable actuatable components such as hands 102 a and/or 102 b).Action(s) selected by a user/pilot of high-level teleoperation interface190 through the GUI are converted into high-level teleoperationinstructions that, when executed by processor 130, cause robot body 101(and any applicable actuatable components such as hands 102 a and/or 102b) to perform the selected action(s).

Teleoperation system 170 is also shown as including at least onenon-transitory processor-readable storage medium (memory) 172, whichoptionally stores reusable work primitives for access by the robot. Thisis particularly useful when memory 140 does not store an entire catalogof reusable work primitives usable by the robot. In such animplementation, the entire catalog of reusable work primitives usable bythe robot can be stored on the memory 172, for access by the robot asneeded. As discussed above, the catalog of reusable work primitives isorganized into libraries of work primitives, where the robot can accesslibraries of reusable work primitives as needed. As needed, at least onelibrary of reusable work primitives can be transferred fromteleoperation system 170 to the robot via wireless transceivers 171 and150, and stored on memory 140.

Teleoperation system 170 can be implemented in a distributed manner. Forexample, memory 172 can be at a server location remote from low-levelteleoperation interface 180 and/or remote from high-level teleoperationinterface 190. As another example, low-level teleoperation interface 180can be remote from high-level teleoperation interface 190.

Robot system 100 in FIG. 1 is illustrated such that the robot generallyemulates or mimics human anatomy. However, this is not necessarily thecase, and any appropriate form of robot could be used. In someimplementations, a robot may only partially emulate human anatomy (e.g.the robot may only include a limited subset of human-like features), ora robot may not emulate human anatomy at all.

FIG. 2 is a flow diagram showing an exemplary method 200 of operation ofa robot. In general, throughout this specification and the appendedclaims, a method of operation of a robot is a method in which at leastsome, if not all, of the various acts are performed by the robot. Forexample, certain acts of a method of operation of a robot may beperformed by at least one processor or processing unit (hereafter“processor”) of the robot communicatively coupled to a non-transitoryprocessor-readable storage medium of the robot and, in someimplementations, certain acts of a method of operation of a robot may beperformed by peripheral components of the robot that are communicativelycoupled to the at least one processor, such as one or more physicallyactuatable components (e.g., arms, legs, end effectors, grippers,hands), one or more sensors (e.g., optical sensors, audio sensors,tactile sensors, haptic sensors), mobility systems (e.g., wheels, legs),communications and networking hardware (e.g., receivers, transmitters,transceivers), and so on. The non-transitory processor-readable storagemedium of the robot may store data (including, e.g., at least onelibrary of reusable work primitives) and/or processor-executableinstructions that, when executed by the at least one processor, causethe robot to perform the method and/or cause the at least one processorto perform those acts of the method that are performed by the at leastone processor. The robot may communicate, via communications andnetworking hardware communicatively coupled to the robot's at least oneprocessor, with remote systems and/or remote non-transitoryprocessor-readable storage media. Thus, unless the specific contextrequires otherwise, references to a robot's non-transitoryprocessor-readable storage medium, as well as data and/orprocessor-executable instructions stored in a non-transitoryprocessor-readable storage medium, are not intended to be limiting as tothe physical location of the non-transitory processor-readable storagemedium in relation to the at least one processor of the robot and therest of the robot hardware. In other words, a robot's non-transitoryprocessor-readable storage medium may include non-transitoryprocessor-readable storage media located on-board the robot and/ornon-transitory processor-readable storage media located remotely fromthe robot, unless the specific context requires otherwise. Further, amethod of operation of a robot such as method 200 (or any of the othermethods discussed herein) can be implemented as a computer programproduct. Such a computer program product comprises processor-executableinstructions or data that, when the computer program product is storedon a non-transitory processor-readable storage medium of the robot, andthe computer program product is executed by at least one processor ofthe robot, the computer program product (or the processor-executableinstructions or data thereof) cause the robot to perform acts of themethod.

Returning to FIG. 2 , method 200 as illustrated includes four acts 201,202, 203, and 204, though those of skill in the art will appreciate thatin alternative implementations certain acts may be omitted and/oradditional acts may be added. Those of skill in the art will alsoappreciate that the illustrated order of the acts is shown for exemplarypurposes only and may change in alternative implementations.

At 201, the robot (e.g. by the at least one processor 130) identifies afirst work objective. In some implementations, the robot may identifythe first work objective in response to receiving instructions fromanother party, such as verbal instructions from a remote or localcontroller or operator. In some implementations, the robot may include acommunication interface (e.g. wireless transceiver 150) communicativelycoupled to the at least one processor and the identification, by therobot, of the first work objective at 201 may include receiving, by thecommunication interface of the robot (e.g., from a remote operator ortele-operation system), instructions related to the first workobjective. Such instructions may include a definition of the first workobjective, parameters dictating how/when/where the first work objectiveshould be completed, and/or directions on how to carry out the firstwork objective.

In other implementations, the identification, by the robot, of the firstwork objective at 201 may include the robot itself (e.g. by the at leastone processor 130) autonomously identifying the first work objective. Insuch implementations, the non-transitory processor-readable storagemedium of the robot may store data, models, policies, paradigms,algorithms, frameworks, architectures, and/or processor-executableinstructions (collectively, “artificial intelligence”) that, whenexecuted by the at least one processor of the robot, cause the robot toautonomously identify the first work objective. The artificialintelligence may autonomously identify the first work objective based ona range of different parameters and/or criteria, including withoutlimitation: sensor data, environmental factors, internal parameters,observations, and/or communications with other systems, robots, devices,or people. As an example, the robot can include at least one sensor(such as the at least one sensor 103 in FIG. 1 ), which captures sensordata representing an environment of the robot (such as visual datarepresenting the environment of the robot, or location data representinga location of the robot, as non-limiting examples). Based on the sensordata, the at least one processor of the robot can identify the firstwork objective. For example, if the sensor data is representative of therobot being in a bathroom, the robot may identify the first workobjective as “clean the bathroom”. The identification of the first workobjective by the robot does not have to be limited to being based on thesensor data, and can be based on additional data or information. Forexample, after identifying that the robot is in a bathroom, the robotmay access cleaning history for the bathroom (e.g. by querying a robotmanagement server such as teleoperation system 170). If the cleaninghistory indicates that the bathroom was recently cleaned (e.g. withinthe last hour or few hours), the robot may identify the first workobjective as “Check the state of the bathroom”. In this way, the robotuses additional context data (cleaning history) to determine the mostappropriate work objective.

Context data received by the communication interface can be used by therobot to identify the first work objective in many situations, evenindependently of sensor data collected by at least one sensor of therobot. As a non-limiting example, a fleet of robots may be utilized toclean and maintain a building. When identifying a work objective toperform (e.g. the first work objective), a robot may access a cleaninghistory database, which indicates a cleaning history for a variety ofareas or cleaning tasks for the building. Based on the database, therobot may identify a first work objective based on what areas or tasksare in greatest need of being taken care of (e.g. what areas or taskshave not been taken care of in the longest amount of time). As anothernon-limiting example, a robot may have access to weather information ascontext data. Such weather information may indicate that it is presentlyraining or rained recently. Based on this weather context data, therobot may determine that the floors of a building or area in the care ofthe robot need to be cleaned.

At 202, the robot accesses a first library of reusable work primitivesfrom a catalog of libraries of reusable work primitives. As discussedearlier, a catalog of work primitives includes all of the reusable workprimitives available and usable by the robot. A catalog of libraries ofreusable work primitives refers to the catalog of work primitives, wherethe reusable work primitives are organized into a plurality of librariesof work primitives. Each library (including the first library ofreusable work primitives) includes fewer reusable work primitives thanthe catalog of libraries of reusable work primitives (though there canbe overlap between reusable primitives included in each library asdiscussed above).

It is preferable that libraries of reusable work primitives which therobot needs are stored locally at the robot (e.g. on memory 140). Tothis end, libraries relevant to a service category which a robot isexpected or intended to operate in, or libraries which the robot mightforeseeably need during a deployment, can be pre-loaded onto the robotprior to deployment. In such a case, accessing the first library ofreusable work primitives from the catalog of libraries of reusable workprimitives as at 202 comprises accessing the first library of reusablework primitives stored on the at least one non-transitoryprocessor-readable storage medium of the robot (e.g. memory 140).However, if at 202 the first library of reusable work primitives whichis needed is not stored locally at the robot, accessing the firstlibrary of reusable work primitives from the catalog of libraries ofreusable work primitives can comprise accessing via the communicationinterface of the robot (e.g. wireless transceiver 150), the firstlibrary of reusable work primitives stored on at least one remotenon-transitory processor-readable storage medium of a device remote fromthe robot (e.g. memory 172 of teleoperation device 170). That is,libraries of reusable work primitives can be stored remotely from therobot, such as on a teleoperation device or on a server accessible tothe robot, and the robot can access libraries of reusable workprimitives as needed. This enables flexibility in the event that theobjectives or tasks faced by the robot don't align with the reusablework primitives it was pre-loaded with. Further, accessing the firstlibrary of reusable work primitives stored on at least one remotenon-transitory processor-readable storage medium of a device remote fromthe robot can comprise transferring, via the communication interface ofthe robot, the first library of reusable work primitives to the at leastone non-transitory processor-readable storage medium of the robot (e.g.transferring the first library of reusable work primitives from memory172 to memory 140). This provides quick access to the first library ofreusable work primitives. This discussion applies to any library ofreusable primitives needed by the robot (and is not limited to the firstlibrary of reusable work primitives).

At 203, the robot identifies a first workflow to complete the first workobjective identified at 201. The first workflow comprises a first set ofreusable work primitives available in the first library of reusable workprimitives. For example, identifying, by the robot, the first workflowmay include identifying or defining a first set or combination ofreusable work primitives that, when performed by the robot, willcomplete the first work objective. Identifying, by the robot, the firstworkflow may further include sequencing or otherwise arranging the firstset or combination of reusable work primitives into a first permutationof the first set of reusable work primitives. As previously described,at 202 the first library of reusable work primitives is accessed, andthe first workflow identified by the robot at 203 comprises, or consistsof, a first set or combination of reusable work primitives selected fromthe first library of reusable work primitives accessed by the robot.

When the robot includes a communication interface (e.g. wirelesstransceiver 150) communicatively coupled to the at least one processor,the identification, by the robot, of the first workflow at 203 mayinclude receiving the first workflow, an indication of the firstworkflow, or instructions related thereto, by the communicationinterface of the robot (e.g., from a remote operator or tele-operationsystem). Such an indication or instructions may include a definition ofthe first workflow, including for example a first set or combination ofreusable work primitives and, optionally, a permutation of the first setor combination of reusable work primitives to complete the first workobjective.

In other implementations, the identification, by the robot, of the firstworkflow at 203 may include the robot itself autonomously identifying afirst set, combination, and/or permutation of reusable work primitivesthat make up the first workflow. In such implementations, thenon-transitory processor-readable storage medium of the robot may storedata, models, policies, paradigms, algorithms, frameworks,architectures, and/or processor-executable instructions (collectively,“artificial intelligence”) that, when executed by the at least oneprocessor of the robot, cause the robot to autonomously identify thefirst workflow. The artificial intelligence may autonomously identifythe first workflow based on a range of different parameters and/orcriteria, including without limitation: sensor data, environmentalfactors, internal parameters, observations, and/or communications withother systems, robots, devices, or people. As an example, the robot caninclude at least one sensor (such as the at least one sensor 103 in FIG.1 ), which captures sensor data representing an environment of the robot(such as visual data representing the environment of the robot, orlocation data representing a location of the robot, as non-limitingexamples). Based on the sensor data, the at least one processor of therobot can identify the first workflow. For example, if the sensor datais representative of the robot being in front of a mirror, the robot mayidentify the first workflow by organizing a first set of work primitivesto clean the mirror (such as described above). The identification of thefirst workflow by the robot does not have to be limited to being basedon the sensor data, and can be based on additional data or information.For example, in some cases different cleaning products may be used atdifferent times to clean the same objects or areas (i.e., differentproducts may be used in a rotation). As a non-limiting example, sinks ina bathroom may be cleaned several times a day, with most of thecleanings being performed with a soap, and with one or two of thecleanings being performed with bleach. When identifying a workflow toaccomplish the objective of cleaning a sink, a robot or teleoperationsystem which is organizing the first workflow can access time and/ordate data, and the workflow can be defined to cause the robot toidentify the correct cleaning solution, and grab said correct cleaningsolution.

Context data received by the communication interface can be used by therobot to identify the first workflow in many situations, evenindependently of sensor data collected by at least one sensor of therobot. As a non-limiting example, a hauling robot (a robot which movesobjects between locations) may be carrying an object, but may not beable to identify what the object is (e.g. it may be in a non-descriptbox). In such a case, the robot may be able to access context data (e.g.a database stored on a server or teleoperation device) which indicateswhat object the robot is carrying. For example, the robot could accessthe database to determine what object was positioned at a location wherethe object was picked up from. As another example, the robot may be ableto scan an optical code (e.g. barcode or QR code) on the box, and lookup what object corresponds to the scanned code in the database. Based onwhat the object is, the first workflow can be identified to involvemaneuvering the robot to place the object in the correct location forsaid object.

In some implementations, acts 202 and 203 in method 200 can be performedin the reverse order from that illustrated in FIG. 2 . In particular,identifying the first workflow to complete the first work objective at203 can comprise identifying a first permutation of a first combinationof reusable work primitives from the catalog of libraries of reusablework primitives. Based on the determined combination of reusable workprimitives, an appropriate library of reusable work primitives (thefirst library in the case of method 200) is accessed by the robot as in202.

At 204, the robot executes the first workflow identified at 203.Generally, in executing the first workflow at 204, the robot executesthe first set of reusable work primitives in the first library ofreusable work primitives identified in the first workflow at 203. Aspreviously described, in some implementations the robot may be trained,configured, or otherwise operable to perform each reusable workprimitive in the library of reusable work primitives substantiallyautonomously or automatically. For example, the non-transitoryprocessor-readable storage medium of the robot may store respective setsof processor-executable instructions that, when executed by the at leastone processor of the robot, each cause the robot to autonomously performa respective one of the reusable work primitives in the first library ofwork primitives. Thus, when the robot executes the first workflow at204, the robot may do so substantially autonomously or automatically byexecuting the sets of processor-executable instructions that cause orenable the robot to autonomously perform the particular reusable workprimitives in the first set of reusable work primitives that correspondto the first workflow.

For the purpose of illustration, an exemplary implementation of method200 is now described.

As an example, a robot may include or access a non-transitoryprocessor-readable storage medium that stores: i) a first library offive first reusable work primitives: A, B, C, D, and E; and ii) fiverespective sets of processor-executable instructions inst(A), inst(B),inst(C), inst(D), and inst(E) that, when executed by at least oneprocessor of the robot, each cause the robot to autonomously perform arespective one of the first reusable work primitives. At 201 of method200, the robot identifies a first work objective (either autonomously orupon receiving instructions as described previously). At 202 of method200, the robot accesses the first library of reusable work primitivesstored on the non-transitory processor-readable storage medium of therobot.

At 203 of method 200, the robot identifies a first workflow to completethe first work objective (either autonomously or upon receivinginstructions as previously described). The first workflow comprises, orconsists of, a first set of reusable work primitives from the firstlibrary of reusable work primitives arranged in a first order. In thisexample, the first workflow consists of reusable work primitives B, C,and D arranged as:

C->B->D

At 204, the robot autonomously executes the first workflow. That is, atleast one processor of the robot executes processor-executableinstructions inst(C) to cause the robot to autonomously perform reusablework primitive C, then at least one processor of the robot executesprocessor-executable instructions inst(B) to cause the robot toautonomously perform reusable work primitive B, then at least oneprocessor of the robot executes processor-executable instructionsinst(D) to cause the robot to autonomously perform reusable workprimitive D. In this way, the robot completes the first work objective:semi-autonomously if the robot relies on receiving instructions in orderto define the first workflow at 203, or completely autonomously if therobot is operable to autonomously define the first workflow at 203.

In accordance with the present robots, methods, and computer programproducts, the catalog of libraries of reusable work primitives mayinclude a plurality of libraries of reusable work primitives directed torespective service categories. For example, the first library ofreusable work primitives stored or accessed by the robot may comprise,or consist of, all of the genericized activities, steps, or sub-tasksnecessary to enable the robot to complete a multitude of different workobjectives within a specific service category. A “service category”refers generally to the nature or type of work a robot is expected toperform in a given deployment. Non-limiting examples of servicecategories include medical service (e.g. nursing or patient careservices); cleaning service; housekeeping service; warehouse service(e.g. management or movement of objects, items, or stock about or amongwarehouses or storage locations); fetch service (e.g. going to retrieveobjects, items, or people); delivery service (bringing objects, items,or people somewhere); repair service (e.g. fixing or maintainingequipment); painting service; and retail service (e.g. cashier or storemanagement). Each library in the catalog of libraries can be directed toa respective service category selected from available servicescategories, including but not limited to the examples listed above. Arobot can be designed to operate within a specific service category(e.g., a medical service robot may include custom medical hardware or bemade of medically safe or sterilizable material), or a robot may begeneric to multiple service categories. In cases where a generic robotis deployable in different service categories, the robot can bepre-loaded with, or access, at least one library appropriate for theservice category in which the robot is to be deployed. For example,prior to or during deployment for painting services, a robot may beloaded with or access at least one library of reusable work primitiveswhich enable to robot to perform painting related tasks, and/oroperating painting-related tools. Further, deployment in a specificservice category is not necessarily permanent. By changing the at leastone library of reusable work primitives readily available to the robot,a service category in which the robot is intended to operate can also bechanged.

The present robots, methods, and computer program products may realize,or at least approximate, general purpose robots that are capable ofcompleting a wide range of different work objectives in a wide range ofdifferent industries, without overburdening a robot with every possibleaction or work primitive the robot may ever be faced with, but rather byproviding the robot with a targeted and specific library or libraries ofreusable work primitives related to a service category the robot isintended to operate in. As mentioned above, inclusion of extra reusablework primitives exponentially increases a possible number of workflows,and thus exponentially increases processing burden to identify orconstruct workflows. Consequently, it is preferable (though not strictlyrequired) for a library to include the bare minimum reusable workprimitives necessary to enable the robot to complete all expectedworkflows for operation in a service category corresponding to thelibrary. However, in some cases, this could lead to the robot not havingappropriate or optimal reusable work primitives to complete certainedge-case tasks that arise when deployed. Thus, in some implementations,it is preferable to strike a balance between including reusable workprimitives beyond the bare minimum (e.g. reusable work primitives whichmay be needed, even if said reusable work primitives are not certainlygoing to be needed), while still not including reusable work primitiveswhich will most likely not be needed for operation in a particularservice category.

Continuing the example above, the exemplary library of five reusablework primitives may, for example, comprise:

A: measure environmental data

B: move to *position_x*

C: pick up *object_i*

D: put down *object_j*

E: barcode scan *object_k*

and the first work objective may be stated as: “move the green box tothe storage room”. Thus, when the robot performs the first workflowC->B->D, the robot: executes inst(C) where *object_i*=“green box” whichcauses the robot to autonomously (i.e., with no further control orinstruction from another party) pick up the green box; executes inst(B)where *position_x*=“storage room” which causes the robot to autonomouslycarry the green box to the storage room; and executes inst(D) where*object_j*=“green box” which causes the robot to autonomously put downthe green box in the storage room. In this way, the robot completes thefirst work objective by performing the first workflow C->B->D.

Using the same library of five reusable work primitives A, B, C, D, andE, the robot may complete a second work objective. For example, a secondwork objective may be stated as: “make an inventory of the items in thestorage room”. The second work objective may be identified by the robot(per act 311 of method 300 discussed later) either autonomously or inresponse to receiving instructions related to the second work objectivefrom some form of controller or operator. The controller/operator mayinclude a remote tele-operator, an at least semi-autonomoustele-operation system, or a controlling entity (e.g., another robot, ora person) co-located with the robot. A controlling entity co-locatedwith the robot may provide verbal instructions to the robot that therobot is operative to detect and process.

After identification of the second work objective, the robot may accessthe first library of reusable work primitives and identify a secondworkflow to complete the second work objective. In a similar way to thefirst workflow, the second workflow also comprises, or consists of, aset of reusable work primitives from the first library of five reusablework primitives A, B, C, D, and E; however, the second workflow isdifferent from the first workflow in that the second workflow comprisesa second, and different, set (e.g., combination and/or permutation) ofreusable work primitives from the library of five reusable workprimitives A, B, C, D, and E. As previously described, the secondworkflow may be identified by the robot (per act 313 of method 300discussed later) either autonomously or in response to receivinginstructions defining the second workflow from some form of controlleror operator.

In this example, the second workflow consists of reusable workprimitives B, C, D, E arranged as:

B->repeat:[C->E->D]

In accordance with the present robots, methods, and computer programproducts, at least one reusable work primitive may be common in multipleworkflows. In the present example, reusable work primitives B, C, and Dare all commonly included in both the first workflow to complete thefirst work objective and the second workflow to complete the second workobjective.

The exemplary second workflow includes “repeat:[C->E->D]”; therefore, inorder to complete the second work objective the robot must repeatprimitives C->E->D of the second workflow for all items in the storageroom. Thus, when the robot executes the second workflowB->repeat:[C->E->D] (per act 314 of method 300 discussed later), therobot executes inst(B) where *position_x*=“storage room” which causesthe robot to autonomously (i.e., with no further control or instructionfrom another party) go to the storage room and then the robot initiatesa loop wherein the robot executes primitives C->E->D for all items inthe storage room. That is, upon arriving in the storage room, the robot:executes inst(C) where *object_i*=“first item” which causes the robot toautonomously pick up a first item in the storage room; executes inst(E)where *object_k*=“first item” which causes the robot to autonomously barcode scan the first item; and executes inst(D) where *object_j*=“firstitem” which causes the robot to put down the first item. The robot thenrepeats primitives C->E->D for each item in the storage room in series(i.e., for a second item, for a third item, and so on) until the robothas picked up, bar code scanned, and put down every item in the storageroom. In this way, the robot completes the second work objective byperforming the workflow B->repeat:[C->E->D].

Using the same first library of five reusable work primitives A, B, C,D, and E, the robot may complete at least one additional work objective.For example, the robot may record environmental parameters at variousstations or waypoints (repeat:[B->A]) and/or checkout customers'purchases at a retail store (repeat:[C->E->D]). These, and many other,work objectives may all be completed by the robot at leastsemi-autonomously by executing corresponding workflows comprising, orconsisting of, various combinations and/or permutations of reusable workprimitives from an exemplary first library of five reusable workprimitives A, B, C, D, and E. However, a person of skill in the art willappreciate that a library of reusable work primitives may consist of anynumber and form of reusable work primitives and the first library offive reusable work primitives A, B, C, D, and E described herein is usedonly as an example for the purpose of illustration. Generally, the morereusable work primitives in a library of reusable work primitives themore different work objectives a robot may be operable tosemi-autonomously complete; however, in some implementations a finitenumber of reusable work primitives (e.g., on the order of 10s, such as10, 20, 30, 40, 50, 60, 70, 80, 90; or on the order of 100s, such as100, 200, etc.) may be sufficient to enable a robot to complete asignificant portion (e.g., all) of the work objectives of interest.

The examples described above, where a robot employs a first library ofreusable work primitives to complete multiple different work objectives,is illustrated in FIG. 3 . FIG. 3 is a flow diagram showing an exemplarymethod 300 of operation of a robot in accordance with the presentrobots, methods, and computer program products. As discussed above withreference to FIG. 2 , a method of operation of a robot is a method inwhich at least some, if not all, of the various acts are performed bythe robot. For example, certain acts of a method of operation of a robotmay be performed by at least one processor or processing unit (hereafter“processor”) of the robot communicatively coupled to a non-transitoryprocessor-readable storage medium of the robot and, in someimplementations, certain acts of a method of operation of a robot may beperformed by peripheral components of the robot that are communicativelycoupled to the at least one processor, such as one or more physicallyactuatable components (e.g., arms, legs, end effectors, grippers,hands), one or more sensors (e.g., optical sensors, audio sensors,tactile sensors, haptic sensors), mobility systems (e.g., wheels, legs),communications and networking hardware (e.g., receivers, transmitters,transceivers), and so on. The non-transitory processor-readable storagemedium of the robot may store data (including, e.g., at least onelibrary of reusable work primitives) and/or processor-executableinstructions that, when executed by the at least one processor, causethe robot to perform the method and/or cause the at least one processorto perform those acts of the method that are performed by the at leastone processor. The robot may communicate, via communications andnetworking hardware communicatively coupled to the robot's at least oneprocessor, with remote systems and/or remote non-transitoryprocessor-readable storage media. Thus, unless the specific contextrequires otherwise, references to a robot's non-transitoryprocessor-readable storage medium, as well as data and/orprocessor-executable instructions stored in a non-transitoryprocessor-readable storage medium, are not intended to be limiting as tothe physical location of the non-transitory processor-readable storagemedium in relation to the at least one processor of the robot and therest of the robot hardware. In other words, a robot's non-transitoryprocessor-readable storage medium may include non-transitoryprocessor-readable storage media located on-board the robot and/ornon-transitory processor-readable storage media located remotely fromthe robot, unless the specific context requires otherwise. Further, amethod of operation of a robot such as method 300 can be implemented asa computer program product. Such a computer program product comprisesprocessor-executable instructions or data that, when the computerprogram product is stored on a non-transitory processor-readable storagemedium of the robot, and the computer program product is executed by atleast one processor of the robot, the computer program product (or theprocessor-executable instructions or data thereof) cause the robot toperform acts of the method.

Method 300 as illustrated in FIG. 3 includes three sub-methods 200, 310,and 320, and twelve illustrated acts 201, 202, 203, 204, 311, 312, 313,314, 321, 322, 323, and 324, though those of skill in the art willappreciate that in alternative implementations certain acts may beomitted and/or additional acts may be added. Those of skill in the artwill also appreciate that the illustrated order of the acts is shown forexemplary purposes only and may change in alternative implementations.

Method 300 includes, as a sub-method 200, method 200 of operation of arobot to complete a first work objective from FIG. 2 . That is, method300 includes acts 201, 202, 203, and 204 from method 200 of FIG. 2 ,whereby the robot identifies a first work objective (201), accesses afirst library of reusable work primitives from a catalog of libraries ofreusable work primitives (202), identifies a first workflow to completethe first work objective (203), and executes the first workflow (204)all substantially as described for method 200 of FIG. 2 . As describedpreviously, the first workflow comprises, or consists of, a particularcombination and/or permutation of reusable work primitives from a firstlibrary of reusable work primitives stored in or otherwise accessed bythe robot. In accordance with method 200, various ones of the reusablework primitives in the robot's first library of reusable work primitivesare sufficient to enable the robot to complete the first work objective.In sub-method 310, various ones of the reusable work primitives in therobot's first library of reusable work primitives are used and/or reusedby the robot to complete a second work objective via acts 311, 312, 313,and 314 (collectively, sub-method 310), and likewise various ones of thereusable work primitives in the robot's first library of reusable workprimitives are used and/or reused by the robot to complete at least oneadditional work objective at sub-method 320.

Specifically, at 311 of sub-method 310 the robot identifies a secondwork objective that is different from the first work objectiveidentified at 201 of sub-method 200. Similar to as described for act201, at 311 the second work objective may be identified autonomously bythe robot or in response to instructions received from another party(e.g., a controller, pilot, or operator) through any of a variety ofdifferent communication means, including verbally, wirelessly through atelecommunications system, digitally through a tethered communicationline, and so on.

At 312 of sub-method 310 the robot accesses the first library ofreusable work primitives from the catalog of libraries of reusable workprimitives. Similar to as described for act 202, at 312 the firstlibrary of reusable work primitives is preferably accessed at a localmemory of the robot, or could be accessed at a remote memory from therobot which the robot has access to. In some cases act 312 could beperformed as (or together with) act 202; that is the first library ofreusable work primitives can be accessed simultaneously, for identifyinga plurality of workflows and/or completing a plurality of workobjectives.

At 313 of sub-method 310, the robot identifies a second workflow tocomplete the second work objective identified at 311. Since the secondwork objective is different from the first work objective, the secondworkflow may be different from the first workflow. Similar to asdescribed for act 203, at 313 identifying, by the robot, the secondworkflow may include identifying or defining a second set or combinationof reusable work primitives that, when performed by the robot, willcomplete the second work objective, and may further include (ifapplicable) sequencing or otherwise arranging the second set orcombination of reusable work primitives into a second permutation of thesecond combination of reusable work primitives. In accordance withmethod 300, the second set or combination of reusable work primitives tocomplete the second work objective is selected, at 313, from the samefirst library of reusable work primitives from which the first set orcombination of reusable work primitives to complete the first workobjective is selected at 203. In some implementations, at least onecommon reusable work primitive may be included in both the firstworkflow identified at 203 and the second workflow identified at 313. Inother words, in some implementations at least one reusable workprimitive from the first workflow identified at 203 is reused in thesecond workflow identified at 313.

Similar to act 203, at 313 the robot may identify the second workflowautonomously or in response to receiving instructions from another party(e.g., a controller, pilot, or operator) through any of a variety ofdifferent communication means, including verbally, wirelessly through atelecommunications system, digitally through a tethered communicationline, and so on.

At 314 of sub-method 310, the robot executes the second workflowidentified at 313. Generally, in executing the second workflow at 314,the robot executes the second set of reusable work primitives selectedfrom the first library of reusable work primitives in the identificationof the second workflow at 313, including (if applicable) at least onecommon reusable work primitive that is included in both the firstworkflow (identified at 203 and executed at 204) and the second workflow(identified at 313 and executed at 314). As previously described, insome implementations the robot may be trained, configured, or otherwiseoperable to perform each reusable work primitive in the first library ofreusable work primitives substantially autonomously or automatically.Thus, when the robot executes the second workflow at 314, the robot maydo so substantially autonomously or automatically by executing theprocessor-executable instructions that cause or enable the robot toautonomously perform the particular reusable work primitives in thesecond set of reusable work primitives that correspond to the secondworkflow. In implementations where both the first workflow identified at203 and the second workflow identified at 313 include at least onecommon reusable work primitive, executing the first workflow at 204 andexecuting the second workflow at 314 may both include executing, by atleast one processor of the robot, at least a same portion ofprocessor-executable instructions that cause the robot to perform the atleast one common reusable work primitive in both the first workflow andthe second workflow.

In accordance with the present robots, methods, and computer programproducts, various ones of the reusable work primitives in the robot'sfirst library of reusable work primitives may be further used and/orreused by the robot to complete at least one additional work objective.To this end, method 300 includes sub-method 200 where a robot identifiesand executes a first set of reusable work primitives to complete a firstwork objective, sub-method 310 where the robot identifies and executes asecond set of reusable work primitives to complete a second workobjective that is different from the first work objective, andsub-method 320 where the robot identifies and executes at least oneadditional set of reusable work primitives to complete at least oneadditional work objective that is different from both the first workobjective and the second work objective.

In sub-method 320, various ones of the reusable work primitives in therobot's first library of reusable work primitives are used and/or reusedby the robot to complete the at least one additional work objective viaacts 321, 322, 323, and 324 (collectively, sub-method 320).Specifically, at 321 of sub-method 320 the robot identifies anadditional work objective that is different from the first workobjective identified at 201 of sub-method 200, and is different from thesecond work objective identified at 311 of sub-method 310. Similar to asdescribed for act 201, at 321 the at least one additional work objectivemay be identified autonomously by the robot or in response toinstructions received from another party (e.g., a controller, pilot, oroperator) through any of a variety of different communication means,including verbally, wirelessly through a telecommunications system,digitally through a tethered communication line, and so on.

At 322 of sub-method 320 the robot accesses the first library ofreusable work primitives from the catalog of libraries of reusable workprimitives. Similar to as described for act 202, at 322 the firstlibrary of reusable work primitives is preferably accessed at a localmemory of the robot, or could be accessed at a remote memory from therobot which the robot has access to. In some cases act 322 could beperformed as (or together with) act 202; that is the first library ofreusable work primitives can be accessed simultaneously, for identifyinga plurality of workflows and/or completing a plurality of workobjectives.

At 323 of sub-method 320, the robot identifies at least one additionalworkflow to complete the at least one additional work objectiveidentified at 321. Since the at least one additional work objective isdifferent from the first work objective and the second work objective,the at least one additional workflow may be different from the firstworkflow and the second workflow. Similar to as described for act 203,at 323 identifying, by the robot, the at least one additional workflowmay include identifying or defining at least one additional set orcombination of reusable work primitives that, when performed by therobot, will complete the at least one additional work objective, and mayfurther include (if applicable) sequencing or otherwise arranging the atleast one additional set or combination of reusable work primitives intoat least one additional permutation of the at least one additionalcombination of reusable work primitives. In accordance with method 300,the at least one additional set or combination of reusable workprimitives to complete the at least one additional work objective isselected, at 323, from the same first library of reusable workprimitives from which the first set or combination of reusable workprimitives to complete the first work objective is selected at 203 andthe second set or combination of reusable work primitives to completethe second work objective are selected at 313. In some implementations,at least one common reusable work primitive may be included in the firstworkflow identified at 203, the second workflow identified at 313,and/or the at least one additional workflow identified at 323. In otherwords, in some implementations at least one reusable work primitive fromthe first workflow identified at 203 or the second workflow identifiedat 313 is reused in the at least one additional workflow identified at323.

Similar to act 203, at 323 the robot may identify the at least oneadditional workflow autonomously or in response to receivinginstructions from another party (e.g., a controller, pilot, or operator)through any of a variety of different communication means, includingverbally, wirelessly through a telecommunications system, digitallythrough a tethered communication line, and so on.

At 324 of sub-method 320, the robot executes the at least one additionalworkflow identified at 323. Generally, in executing the at least oneadditional workflow at 324, the robot executes the at least oneadditional set of reusable work primitives selected from the firstlibrary of reusable work primitives in the identification of the atleast one additional workflow at 323, including (if applicable) at leastone common reusable work primitive that is included in the firstworkflow (identified at 203 and executed at 204) or the second workflow(identified at 313 and executed at 314), and the at least one additionalworkflow (identified at 323 and executed at 324). As previouslydescribed, in some implementations the robot may be trained, configured,or otherwise operable to perform each reusable work primitive in thefirst library of reusable work primitives substantially autonomously orautomatically. Thus, when the robot executes the at least one additionalworkflow at 324, the robot may do so substantially autonomously orautomatically by executing the processor-executable instructions thatcause or enable the robot to autonomously perform the particularreusable work primitives in the at least one additional set of reusablework primitives that correspond to the at least one additional workflow.In implementations where the first workflow identified at 203, thesecond workflow identified at 313, or the at least one additionalworkflow identified at 323 include at least one common reusable workprimitive, executing the first workflow at 204, executing the secondworkflow at 314, and executing the at least one additional workflow at324 may include executing, by at least one processor of the robot, atleast a same portion of processor-executable instructions that cause therobot to perform the at least one common reusable work primitive.

Sub-method 320 in FIG. 3 can be repeated as many times as isappropriate, to complete a plurality of different additional workobjectives, by identifying and executing a plurality of differentadditional workflows.

In each of sub-methods 200, 310, and 320, the same robot is deployedusing the same first library of reusable work primitives; however,because the first work objective, the second work objective, and the atleast one additional work objective are all different from one another,the corresponding workflows (i.e., the first workflow, the secondworkflow, and the at least one additional workflow, respectively)identified by the robot to complete the work objectives may all bedifferent from another. In accordance with the present robots, methods,and computer program products, the work primitives in the robot's firstlibrary of work primitives are reusable such that any or all of thefollowing is/are true: i) at least one work primitive that is includedin the first workflow to complete the first work objective can also beincluded in (i.e., can be common to or “reused” in) the second workflowto complete the second work objective; ii) at least one work primitivethat is included in the second workflow to complete the second workobjective can also be included in (i.e., can be common to or “reused”in) the at least one additional workflow to complete the at least oneadditional work objective; iii) at least one work primitive that isincluded in the at least one additional workflow to complete the atleast one additional work objective can also be included in (i.e., canbe common to or reused in) the first workflow to complete the first workobjective; and/or iv) at least one work primitive that is included inthe first workflow to complete the first work objective can also beincluded in (i.e., can be common to or reused in) both the secondworkflow to complete the second work objective and the at least oneadditional workflow to complete the at least one additional workobjective. In the latter scenario, at least one reusable work primitiveis included in (i.e., is common to or reused in) all of the firstworkflow, the second workflow, and the at least one additional workflow.While it is possible for the first workflow, the second workflow, and/orthe third workflow to include at least one common reusable workprimitive, this is not necessarily required. Rather, the first libraryof reusable work primitives is generally relevant to a specific servicecategory or role in which the robot is deployed. There may be a greatmany work objectives which a robot is expected to perform within a givenservice category or role. Some of these work objectives may beaccomplished by workflows with overlapping reusable work primitives, butsome work objectives may be accomplished by workflows which use acomplete different set of reusable work primitives from other workflows.A library corresponding to a particular service category can includesufficient reusable work primitives such that even very differentworkflows (e.g. workflows which do not share common reusable workprimitives), directed to work objectives within the service category,can still be identified and executed by a robot based on the library. Asan example, a robot deployed in a medical service category may execute aworkflow to fill a syringe with a medication, and also execute aworkflow to push a gurney from one location to another. Even thoughthese two tasks may not share a common reusable work primitive, thelibrary for the service category still includes reusable work primitivesto accomplish both tasks.

A robot may come in a variety of different forms and include a varietyof different components. A robot may include a body or chassis thatincludes, houses, carries, or is otherwise mechanically coupled to, aspreviously described, at least one processor communicatively coupled toat least one non-transitory processor-readable storage medium.Advantageously, a robot may also include at least one physicallyactuatable component mechanically coupled to the body andcommunicatively coupled to the at least one processor, the at least onephysically actuatable component operable to controllably move and effectchanges in, on, or to the robot and/or the robot's environment.Throughout this specification and the appended claims, the term“physically actuatable component” refers to a real physical part of arobot that is controllably actuatable (e.g., by the robot, or by anoperator, controller or pilot of the robot) to change in some physicalway involving motion (including, without limitation, translation and/orrotation). Non-limiting examples of physically actuatable componentsinclude: an arm, an end effector, a gripper, a hand, a finger, a leg, anextendible and/or displaceable support structure such as a neck or boom,and a wheel.

In accordance with the present robots, methods, and computer programproducts, when a robot includes at least a first physically actuatablecomponent, a catalog of libraries of reusable work primitives caninclude libraries of reusable work primitives performable by the firstphysically actuatable component. Further, the first library of reusablework primitives stored in (e.g., the at least one non-transitoryprocessor-readable storage medium of the robot) or otherwise accessed bythe robot may include a set of reusable work primitives performable byat least the first physically actuatable component. That is, the robot'sreusable work primitives accessible to the robot may include specificreusable work primitives that are each performable by at least the firstphysically actuatable component.

Method 300 in FIG. 3 involves executing a plurality of workflows tocomplete a plurality of work objectives, based on reusable workprimitives from a first library of reusable work primitives. Whendeploying a robot to perform in a given service category or role, therobot is loaded with the first library of work primitives (the firstlibrary of work primitives is stored on a non-transitoryprocessor-readable storage medium of the robot), or the first library ismade accessible to the robot (e.g. by being stored on a non-transitoryprocessor-readable storage medium of a server accessible to the robot).Preferably, the first library of reusable work primitives includes everywork primitive (or includes a plurality of sub-libraries which togetherinclude every work primitive) that the robot could foreseeably need tosatisfactorily perform in the service category or role in which therobot is deployed. However, as discussed earlier, the first library ofreusable work primitives does not include every work primitive which therobot is capable of using, to save processing or storage resources atthe robot. Rather, the first library comprises a subset of reusable workprimitives of a catalog of reusable work primitives usable by the robot.In some situations, a robot may need to access at least one reusablework primitive which is not included in the library (e.g. to act in anunforeseen or uncommon situation, or when the robot is being re-deployedin a different service category or role). FIG. 4 discussed below detailsa method of accomplishing different work objectives using differentlibraries of reusable work primitives.

FIG. 4 is a flow diagram showing an exemplary method 400 of operation ofa robot in accordance with the present robots, methods, and computerprogram products. Method 400 includes three sub-methods 200, 410, and420, and twelve illustrated acts 201, 202, 203, 204, 411, 412, 413, 414,421, 422, 423, and 424, though those of skill in the art will appreciatethat in alternative implementations certain acts may be omitted and/oradditional acts may be added. Those of skill in the art will alsoappreciate that the illustrated order of the acts is shown for exemplarypurposes only and may change in alternative implementations.

Method 400 includes, as a sub-method 200, method 200 of operation of arobot to complete a first work objective from FIG. 2 . That is, method400 includes acts 201, 202, 203, and 204 from method 200 of FIG. 2 ,whereby the robot identifies a first work objective (201), accesses afirst library of reusable work primitives from a catalog of libraries ofreusable work primitives (202), identifies a first workflow to completethe first work objective (203), and executes the first workflow (204)all substantially as described for method 200 of FIG. 2 . As describedpreviously, the first workflow comprises, or consists of, a particularcombination and/or permutation of reusable work primitives from a firstlibrary of reusable work primitives stored in or otherwise accessed bythe robot. In accordance with method 200, various ones of the reusablework primitives in the robot's first library of reusable work primitivesare sufficient to enable the robot to complete the first work objective.

In sub-method 410, reusable work primitives in a second library ofreusable work primitives accessible to the robot are used and/or reusedby the robot to complete a second work objective via acts 411, 412, 413,and 414 (collectively, sub-method 410), and likewise reusable workprimitives in at least one additional library of reusable workprimitives accessible to the robot are used and/or reused by the robotto complete at least one additional work objective at sub-method 420.

Specifically, at 411 of sub-method 410 the robot identifies a secondwork objective that is different from the first work objectiveidentified at 201 of sub-method 200. Similar to as described for act201, at 411 the second work objective may be identified autonomously bythe robot or in response to instructions received from another party(e.g., a controller, pilot, or operator) through any of a variety ofdifferent communication means, including verbally, wirelessly through atelecommunications system, digitally through a tethered communicationline, and so on.

At 412 of sub-method 410 the robot accesses a second library of reusablework primitives from the catalog of libraries of reusable workprimitives. Like the first library of reusable work primitives, thesecond library of reusable work primitives includes fewer reusable workprimitives than the catalog of libraries of reusable work primitives.The second library of reusable work primitives is at least partiallydifferent from the first library of reusable work primitives asdiscussed earlier. Similar to as described for act 202, at 412 thesecond library of reusable work primitives could be accessed at a localmemory of the robot, or could be accessed at a remote memory from therobot which the robot has access to.

At 413 of sub-method 410, the robot identifies a second workflow tocomplete the second work objective identified at 411. Since the secondwork objective is different from the first work objective, the secondworkflow may be different from the first workflow. Similar to asdescribed for act 203, at 413 identifying, by the robot, the secondworkflow may include identifying or defining a second set or combinationof reusable work primitives that, when performed by the robot, willcomplete the second work objective, and may further include (ifapplicable) sequencing or otherwise arranging the second set orcombination of reusable work primitives into a second permutation of thesecond combination of reusable work primitives. In accordance withmethod 400, the second set or combination of reusable work primitives tocomplete the second work objective is selected, at 413, from the secondlibrary of reusable work primitives accessed at 412. In someimplementations, at least one common reusable work primitive may beincluded in both the first workflow identified at 203 and the secondworkflow identified at 413. In other words, in some implementations atleast one reusable work primitive from the first workflow identified at203 is reused in the second workflow identified at 413.

Similar to act 203, at 413 the robot may identify the second workflowautonomously or in response to receiving instructions from another party(e.g., a controller, pilot, or operator) through any of a variety ofdifferent communication means, including verbally, wirelessly through atelecommunications system, digitally through a tethered communicationline, and so on.

At 414 of sub-method 410, the robot executes the second workflowidentified at 413. Generally, in executing the second workflow at 414,the robot executes the second set of reusable work primitives selectedfrom the second library of reusable work primitives in theidentification of the second workflow at 413. As previously described,in some implementations the robot may be trained, configured, orotherwise operable to perform each reusable work primitive in the secondlibrary of reusable work primitives substantially autonomously orautomatically. Thus, when the robot executes the second workflow at 414,the robot may do so substantially autonomously or automatically byexecuting the processor-executable instructions that cause or enable therobot to autonomously perform the particular reusable work primitives inthe second set of reusable work primitives that correspond to the secondworkflow. In implementations where both the first workflow identified at203 and the second workflow identified at 413 include at least onecommon reusable work primitive, executing the first workflow at 204 andexecuting the second workflow at 414 may both include executing, by atleast one processor of the robot, at least a same portion ofprocessor-executable instructions that cause the robot to perform the atleast one common reusable work primitive in both the first workflow andthe second workflow.

In accordance with the present robots, methods, and computer programproducts, reusable work primitives in at least one additional library ofreusable work primitives may be further used and/or reused by the robotto complete at least one additional work objective. To this end, method400 further includes sub-method 420 where the robot identifies andexecutes at least one additional set of reusable work primitives tocomplete at least one additional work objective that is different fromboth the first work objective and the second work objective.

In sub-method 420, reusable work primitives in at least one additionallibrary of reusable work primitives are used and/or reused by the robotto complete the at least one additional work objective via acts 421,422, 423, and 424 (collectively, sub-method 420). Specifically, at 421of sub-method 420 the robot identifies an additional work objective thatis different from the first work objective identified at 201 ofsub-method 200, and is different from the second work objectiveidentified at 411 of sub-method 410. Similar to as described for act201, at 421 the at least one additional work objective may be identifiedautonomously by the robot or in response to instructions received fromanother party (e.g., a controller, pilot, or operator) through any of avariety of different communication means, including verbally, wirelesslythrough a telecommunications system, digitally through a tetheredcommunication line, and so on.

At 422 of sub-method 420 the robot accesses at least one additionallibrary of reusable work primitives from the catalog of libraries ofreusable work primitives. Like the first library of reusable workprimitives and the second library of reusable work primitives, the atleast one additional library of reusable work primitives includes fewerreusable work primitives than the catalog of libraries of reusable workprimitives. Similar to as described for act 202, at 422 the at least oneadditional library of reusable work primitives can be accessed at alocal memory of the robot, or could be accessed at a remote memory fromthe robot which the robot has access to.

At 423 of sub-method 420, the robot identifies at least one additionalworkflow to complete the at least one additional work objectiveidentified at 421. Since the at least one additional work objective isdifferent from the first work objective and the second work objective,the at least one additional workflow may be different from the firstworkflow and the second workflow. Similar to as described for act 203,at 423 identifying, by the robot, the at least one additional workflowmay include identifying or defining at least one additional set orcombination of reusable work primitives that, when performed by therobot, will complete the at least one additional work objective, and mayfurther include (if applicable) sequencing or otherwise arranging the atleast one additional set or combination of reusable work primitives intoat least one additional permutation of the at least one additionalcombination of reusable work primitives. In accordance with method 400,the at least one additional set or combination of reusable workprimitives to complete the at least one additional work objective isselected, at 423, from the at least one additional library of reusablework primitives accessed at 422. In some implementations, at least onecommon reusable work primitive may be included in the first workflowidentified at 203, the second workflow identified at 413, and/or the atleast one additional workflow identified at 423. In other words, in someimplementations at least one reusable work primitive from the firstworkflow identified at 203 or the second workflow identified at 413 isreused in the at least one additional workflow identified at 423.

Similar to act 203, at 423 the robot may identify the at least oneadditional workflow autonomously or in response to receivinginstructions from another party (e.g., a controller, pilot, or operator)through any of a variety of different communication means, includingverbally, wirelessly through a telecommunications system, digitallythrough a tethered communication line, and so on.

At 424 of sub-method 420, the robot executes the at least one additionalworkflow identified at 423. Generally, in executing the at least oneadditional workflow at 424, the robot executes the at least oneadditional set of reusable work primitives selected from the at leastone additional library of reusable work primitives in the identificationof the at least one additional workflow at 423. As previously described,in some implementations the robot may be trained, configured, orotherwise operable to perform each reusable work primitive in the atleast one additional library of reusable work primitives substantiallyautonomously or automatically. Thus, when the robot executes the atleast one additional workflow at 424, the robot may do so substantiallyautonomously or automatically by executing the processor-executableinstructions that cause or enable the robot to autonomously perform theparticular reusable work primitives in the at least one additional setof reusable work primitives that correspond to the at least oneadditional workflow. In implementations where the first workflowidentified at 203, the second workflow identified at 413, or the atleast one additional workflow identified at 423 include at least onecommon reusable work primitive, executing the first workflow at 204,executing the second workflow at 414, and executing the at least oneadditional workflow at 424 may include executing, by at least oneprocessor of the robot, at least a same portion of processor-executableinstructions that cause the robot to perform the at least one commonreusable work primitive.

Sub-method 420 in FIG. 4 can be repeated as many times as isappropriate, to complete a plurality of different additional workobjectives, by identifying and executing a plurality of differentadditional workflows.

In an exemplary use case, if a deployed robot finds itself unable toaccomplish a work objective (the second work objective) using the firstlibrary of reusable work primitives (preferably stored on a memory ofthe robot), the robot can access the second library of reusable workprimitives, identify a second workflow, and execute the second workflowper sub-method 410 in FIG. 4 . Further, if the deployed robot findsitself unable to accomplish another work objective (the at least oneadditional work objective) using the first library of reusable workprimitives or the second library of work objectives, the robot canaccess the at least one additional library of reusable work primitives,identify at least one additional workflow, and execute the at least oneadditional workflow per sub-method 420 in FIG. 4 .

In another exemplary use case, a robot may be deployed in a firstservice category, where the robot generally accesses the first libraryof reusable work primitives, identifies work objectives and workflows,and executes workflows based on the first library of reusable workprimitives, in accordance with method 200 in FIG. 2 (or sub-method 200in FIG. 4 ). Said robot may be redeployed to a second service categorydifferent from the first service category, where expectations andobjectives of the robot will be different from the first servicecategory. During deployment in the second service category, the robotgenerally accesses a second library of reusable work primitives,identifies work objectives and workflows, and executes workflows basedon the second library of reusable work primitives, in accordance withsub-method 410 in FIG. 4 . Further, said robot may be further redeployedto at least one additional service category different from the firstservice category and the second service category, where expectations andobjectives of the robot will be further different from the first servicecategory and the second service category. During deployment in the atleast one additional service category, the robot generally accesses atleast one additional library of reusable work primitives, identifieswork objectives and workflows, and executes workflows based on the atleast one additional library of reusable work primitives, in accordancewith sub-method 420 in FIG. 4 .

FIG. 4 illustrates the ability for the robot to operate in differentservice categories, based on different libraries of work primitivesselected based on a pertinent service category for deployment. All ofthe sub-methods 200, 410, and 420 are not necessarily required for aparticular robot.

The completion of some work objectives may advantageously deploy arobot's first physically actuatable component, e.g., in order to effecta change on the robot and/or its environment. In this case, when therobot identifies a workflow to complete a work objective per act 203 ofmethod 200, act 313 of (sub-method 310 of) method 300, act 323 of(sub-method 320 of) method 300, act 413 of (sub-method 410 of) method400, and/or act 423 of (sub-method 410 of) method 400, the robot mayidentify a set of reusable work primitives that includes at least onereusable work primitive from the set of reusable work primitivesperformable by the first physically actuatable component. When the robotexecutes a workflow, per act 204 of method 200, act 314 of (sub-method310 of) method 300, act 324 of (sub-method 320 of) method 300, act 414of (sub-method 410 of) method 400, and/or act 424 of (sub-method 410 of)method 400, the robot executes or performs the at least one reusablework primitive from the set of reusable work primitives performable bythe first physically actuatable component, which results in the firstphysically actuatable component actuating to effect a change on therobot or the robot's environment in some deliberate way characterized bythe at least one reusable work primitive performable by the firstphysically actuatable component. Specific examples of the above forwhich the at least one physically actuatable component is or includes acomponent (e.g., end effector) operative to grasp objects, such as agripper or hand, are discussed in U.S. patent application Ser. No.17/566,589, which is incorporated herein by reference in its entirety.

Several examples discussed above mention that a library of reusable workprimitives is stored on the robot, and/or accessed by the robot, toexecute a workflow to accomplish a work objective. Said library ofreusable work primitives is ideally as relevant as possible to the workobjective or workflow, or a service category or role in which the robotoperates. To this end, the library (e.g. the first library in method200) is selected from a plurality of libraries of reusable workprimitives (the catalog of libraries of reusable work primitives), whereeach library of reusable work primitives includes a respective pluralityof reusable work primitives which is at least partially different fromrespective pluralities of reusable work primitives included in otherlibraries of reusable work primitives. Several examples are discussedbelow regarding how a library is selected for use in identifying aworkflow and executing the workflow. While the examples reference thefirst library (in method 200, sub-method 200, sub-method 310, orsub-method 320), the discussion applies to selection of any library,such as the second library in sub-method 410 or the at least oneadditional library in sub-method 420.

In a first example, with reference to method 200, the first library ofreusable work primitives is selected from the catalog of libraries ofreusable work primitives based on the first work objective. That is,based on a work objective to be accomplished by the robot, anappropriate library from the catalog of libraries or reusable workprimitives is selected for access and use by the robot.

In a second example, with reference to method 200, the first library ofreusable work primitives is selected from the catalog of libraries ofreusable work primitives based on the first workflow. That is, aworkflow is identified by the robot (e.g. based on all reusable workprimitives usable by the robot, even if all of the reusable workprimitives are not stored at or readily accessible to the robot); basedon the identified workflow, an appropriate library from the catalog oflibraries or reusable work primitives is selected for access and use bythe robot (e.g. the appropriate library can be transferred to the robot,or made accessible to the robot on a server).

In a third example, with reference to method 200, the first library ofreusable work primitives from the catalog of libraries of reusable workprimitives is selected based on a service category. The service categorycould be intrinsic to the robot (e.g. the robot is designed to operatein the service category), or the service category could be assigned to arobot that is otherwise generic to multiple service categories. In oneimplementation, at least one processor of the robot could identify aservice category based on an indication of the service category storedon at least one non-transitory processor-readable storage medium of therobot. For example, prior to or during deployment of the robot,instructions relating to the service category of the robot could betransferred to the non-transitory processor-readable storage medium ofthe robot. In another implementation, where the robot includes acommunication interface communicatively coupled to at least oneprocessor of the robot, the at least one processor or the robot canidentify the service category based on an indication of the servicecategory received via the communication interface. For example,instructions indicating a service category could be sent to the robotfrom a remote device or server, to deploy the robot in the servicecategory (or to redeploy and already deployed robot in a differentservice category). In these exemplary implementations, based on theidentified service category, the robot accesses an appropriate,relevant, or corresponding library of reusable work primitives.

In each of method 200 (or sub method 200), sub method 310, sub-method320, sub-method 410, and sub-method 420, identifying a work objective isillustrated as occurring prior to accessing a library of reusable workprimitives used to complete the work objective. For example, withreference to method 200 in FIG. 2 , identifying the first work objectiveas in 201 can include identifying the first work objective based atleast in part on a service category. However, this is not necessarilythe case. In some cases, accessing a library of reusable work primitivescan occur prior to identifying a work objective, and identification of awork objective is based on the accessed library of reusable workprimitives. For example, with reference to method 200 in FIG. 2 , thefirst library of reusable work primitives can be accessed at 202, thenidentifying the first work objective at 201 can be based on the firstlibrary of reusable work primitives. Such a sequence can be advantageousin situations where a robot is uncertain what it should be doing (e.g.has multiple possible work objectives, or has work objectives which therobot has relatively low confidence are appropriate). In suchsituations, the robot can determine what work objective is mostappropriate, based on what the robot is capable of readily doing (byvirtue of reusable work primitives in the first library accessed by therobot). In such implementations/scenarios, identifying a workflow canalso be based on a library of reusable work primitives accessed by thefirst robot. For example, identifying the first workflow as at 203includes identifying the first workflow based on the first library ofreusable work primitives accessed at 202. Though this discussionspecifically references method 200 in FIG. 2 and acts thereof, a similardiscussion is also applicable to sub-methods 200, 310, 320, 410, and 420in FIGS. 3 and 4 , and is not repeated for brevity.

As discussed earlier, the catalog of libraries of reusable workprimitives includes a plurality of libraries of reusable workprimitives. Each library of reusable work primitives includes arespective plurality of reusable work primitives which is at leastpartially different from respective pluralities of reusable workprimitives included in other libraries of reusable work primitives. Someexamples are discussed below of how different libraries of reusable workprimitives are defined (i.e. how a plurality of reusable work primitivesare grouped together as a library).

In some cases, each library of reusable work primitives is definedaccording to a set of work objectives performable by reusable workprimitives included in the respective library of reusable workprimitives. In a sense, this can be considered as defining a library ofreusable work objectives based on a service category to which thelibrary is directed. For example, for a given service category, certainwork objectives will commonly, generally, or at least occasionally beexpected to be completed by the robot. A service category, and acorresponding library of reusable work primitives, can be defined basedon work objectives expected to be performed when a robot is deployed insuch a service category.

In other cases, each library of reusable work primitives is definedaccording to a place of use for the respective library of reusable workprimitives. That is, in some cases a place of use is indicative of whatactions a robot may need to perform, and thus what reusable workprimitives the robot may need to use. For example, a robot deployed in ahospital may be expected to perform medical activities, and thus canhave access to reusable work primitives related to such medicalactivities.

The above means for defining libraries of reusable work primitives aremerely exemplary, and other means for defining libraries of reusablework primitives could be implemented as appropriate for a givenapplication.

In accordance with the present robots, methods, and computer programproducts, a multi-purpose robot may be trained to autonomously execute afinite number of reusable work primitives in order to autonomouslyperform a multitude of tasks. Such a multi-purpose robot may include anon-board non-transitory processor-readable storage mediumcommunicatively coupled to at least one on-board processor. Thenon-transitory processor-readable storage medium may store data and/orprocessor-executable instructions, which may include a library ofreusable primitives from a catalog of libraries of reusable workprimitives that the robot is capable of performing autonomously. Inoperation, the at least one processor of the robot may execute dataand/or processor-executable instructions to cause the robot to analyze atask and identify a particular sequence of work primitives from thestored library of work primitives that, when carried out by the robot,will result in completion of the task's objective.

The robots described herein may, in some implementations, employ any ofthe teachings of U.S. patent application Ser. No. 16/940,566(Publication No. US 2021-0031383 A1), U.S. patent application Ser. No.17/023,929 (Publication No. US 2021-0090201 A1), U.S. patent applicationSer. No. 17/061,187 (Publication No. US 2021-0122035 A1), U.S. patentapplication Ser. No. 17/098,716 (Publication No. US 2021-0146553 A1),U.S. patent application Ser. No. 17/111,789 (Publication No. US2021-0170607 A1), U.S. patent application Ser. No. 17/158,244(Publication No. US 2021-0234997 A1), U.S. Provisional PatentApplication Ser. No. 63/001,755 (Publication No. US 2021-0307170 A1),and/or U.S. Provisional Patent Application Ser. No. 63/057,461, as wellas U.S. Provisional Patent Application Ser. No. 63/151,044, U.S.Provisional Patent Application Ser. No. 63/173,670, U.S. ProvisionalPatent Application Ser. No. 63/184,268, U.S. Provisional PatentApplication Ser. No. 63/213,385, U.S. Provisional Patent ApplicationSer. No. 63/232,694, U.S. Provisional Patent Application Ser. No.63/253,591, U.S. Provisional Patent Application Ser. No. 63/293,968,U.S. Provisional Patent Application Ser. No. 63/293,973, U.S.Provisional Patent Application Ser. No. 63/278,817, and/or U.S. patentapplication Ser. No. 17/566,589, each of which is incorporated herein byreference in its entirety.

Throughout this specification and the appended claims the term“communicative” as in “communicative coupling” and in variants such as“communicatively coupled,” is generally used to refer to any engineeredarrangement for transferring and/or exchanging information. For example,a communicative coupling may be achieved through a variety of differentmedia and/or forms of communicative pathways, including withoutlimitation: electrically conductive pathways (e.g., electricallyconductive wires, electrically conductive traces), magnetic pathways(e.g., magnetic media), wireless signal transfer (e.g., radio frequencyantennae), and/or optical pathways (e.g., optical fiber). Exemplarycommunicative couplings include, but are not limited to: electricalcouplings, magnetic couplings, radio frequency couplings, and/or opticalcouplings.

Throughout this specification and the appended claims, infinitive verbforms are often used. Examples include, without limitation: “to encode,”“to provide,” “to store,” and the like. Unless the specific contextrequires otherwise, such infinitive verb forms are used in an open,inclusive sense, that is as “to, at least, encode,” “to, at least,provide,” “to, at least, store,” and so on.

This specification, including the drawings and the abstract, is notintended to be an exhaustive or limiting description of allimplementations and embodiments of the present systems, devices, andmethods. A person of skill in the art will appreciate that the variousdescriptions and drawings provided may be modified without departingfrom the spirit and scope of the disclosure. In particular, theteachings herein are not intended to be limited by or to theillustrative examples of computer systems and computing environmentsprovided.

This specification provides various implementations and embodiments inthe form of block diagrams, schematics, flowcharts, and examples. Aperson skilled in the art will understand that any function and/oroperation within such block diagrams, schematics, flowcharts, orexamples can be implemented, individually and/or collectively, by a widerange of hardware, software, and/or firmware. For example, the variousembodiments disclosed herein, in whole or in part, can be equivalentlyimplemented in one or more: application-specific integrated circuit(s)(i.e., ASICs); standard integrated circuit(s); computer program(s)executed by any number of computers (e.g., program(s) running on anynumber of computer systems); program(s) executed by any number ofcontrollers (e.g., microcontrollers); and/or program(s) executed by anynumber of processors (e.g., microprocessors, central processing units,graphical processing units), as well as in firmware, and in anycombination of the foregoing.

Throughout this specification and the appended claims, a “memory” or“storage medium” is a processor-readable medium that is an electronic,magnetic, optical, electromagnetic, infrared, semiconductor, or otherphysical device or means that contains or stores processor data, dataobjects, logic, instructions, and/or programs. When data, data objects,logic, instructions, and/or programs are implemented as software andstored in a memory or storage medium, such can be stored in any suitableprocessor-readable medium for use by any suitable processor-relatedinstruction execution system, apparatus, or device, such as acomputer-based system, processor-containing system, or other system thatcan fetch the data, data objects, logic, instructions, and/or programsfrom the memory or storage medium and perform various acts ormanipulations (i.e., processing steps) thereon and/or in responsethereto. Thus, a “non-transitory processor-readable storage medium” canbe any element that stores the data, data objects, logic, instructions,and/or programs for use by or in connection with the instructionexecution system, apparatus, and/or device. As specific non-limitingexamples, the processor-readable medium can be: a portable computerdiskette (magnetic, compact flash card, secure digital, or the like), arandom access memory (RAM), a read-only memory (ROM), an erasableprogrammable read-only memory (EPROM, EEPROM, or Flash memory), aportable compact disc read-only memory (CDROM), digital tape, and/or anyother non-transitory medium.

The claims of the disclosure are below. This disclosure is intended tosupport, enable, and illustrate the claims but is not intended to limitthe scope of the claims to any specific implementations or embodiments.In general, the claims should be construed to include all possibleimplementations and embodiments along with the full scope of equivalentsto which such claims are entitled.

1. A computer program product comprising processor-executableinstructions or data that, when the computer program product is storedin a non-transitory processor-readable storage medium of a robot, andthe computer program product is executed by at least one processor ofthe robot, the at least one processor communicatively coupled to thenon-transitory processor-readable storage medium, causes the robot to:identify a first work objective; access a first library of reusable workprimitives from a catalog of libraries of reusable work primitives, thefirst library of reusable work primitives including fewer reusable workprimitives than the catalog of libraries of reusable work primitives;identify a first workflow to complete the first work objective, thefirst workflow comprising a first set of reusable work primitivesavailable in the first library of reusable work primitives of thecatalog of libraries of reusable work primitives; and execute the firstworkflow by executing the first set of reusable work primitives in thefirst library of reusable work primitives.
 2. The computer programproduct of claim 1 wherein the processor-executable instructions or dataof the computer program product further cause the robot to: identify asecond work objective different from the first work objective; access asecond library of reusable work primitives from the catalog of librariesof reusable work primitives, the second library of reusable workprimitives including fewer reusable work primitives than the catalog oflibraries of reusable work primitives; identify a second workflow tocomplete the second work objective, the second workflow comprising asecond set of reusable work primitives available in the second libraryof reusable work primitives of the catalog of libraries of reusable workprimitives, the second set of reusable work primitives being at leastpartially different from the first set of reusable work primitives; andexecute the second workflow by executing the second set of reusable workprimitives in the second library of reusable work primitives.
 3. Thecomputer program product of claim 2 wherein the processor-executableinstructions or data of the computer program product further cause therobot to: identify at least one additional work objective different fromthe first work objective and the second work objective; access at leastone additional library of reusable work primitives from the catalog oflibraries of reusable work primitives, each additional library ofreusable work primitives including fewer reusable work primitives thanthe catalog of libraries of reusable work primitives; identify at leastone additional workflow to complete the at least one additional workobjective, the at least one additional workflow comprising at least oneadditional set of reusable work primitives available in the at least oneadditional library of reusable work primitives, the at least oneadditional set of reusable work primitives being at least partiallydifferent from the first set of reusable work primitives and the secondset of reusable work primitives; and execute the at least one additionalworkflow by executing the at least one additional set of reusable workprimitives in the at least one additional library of reusable workprimitives.
 4. The computer program product of claim 1 wherein theprocessor-executable instructions or data of the computer programproduct further cause the robot to: identify a second work objectivedifferent from the first work objective; identify a second workflow tocomplete the second work objective, the second workflow comprising asecond set of reusable work primitives available in the first library ofreusable work primitives of the catalog of libraries of reusable workprimitives, the second set of reusable work primitives being at leastpartially different from the first set of reusable work primitives; andexecute the second workflow by executing the second set of reusable workprimitives in the first library of reusable work primitives.
 5. Thecomputer program product of claim 1, the robot further including acommunication interface communicatively coupled to the at least oneprocessor, and wherein the processor-executable instructions or data ofthe computer program product which cause the robot to identify the firstwork objective cause the robot to: identify the first work objectivebased on instructions related to the first work objective received viathe communication interface.
 6. The computer program product of claim 1,wherein: the robot includes at least one sensor; theprocessor-executable instructions or data of the computer programproduct further cause the robot to capture, by the at least one sensor,sensor data representing an environment of the robot; and theprocessor-executable instructions or data of the computer programproduct which cause the robot to identify the first work objective causethe robot to: identify, by the at least one processor of the robot, thefirst work objective based on at least the sensor data.
 7. The computerprogram product of claim 1, the robot including a communicationinterface communicatively coupled to the at least one processor, whereinthe processor-executable instructions or data of the computer programproduct which cause the robot to identify the first work objective causethe robot to: identify, by the at least one processor of the robot, thefirst work objective based on context data received by the communicationinterface from a device remote from the robot body.
 8. The computerprogram product of claim 1, the robot including a communicationinterface communicatively coupled to the at least one processor, whereinthe processor-executable instructions or data of the computer programproduct which cause the robot to identify the first workflow cause therobot to: identify the first workflow based on an indication related tothe first workflow received via the communication interface.
 9. Thecomputer program product of claim 1 wherein the processor-executableinstructions or data of the computer program product which cause therobot to identify the first workflow cause the robot to: autonomouslyidentify, by the at least one processor of the robot, the first set ofreusable work primitives.
 10. The computer program product of claim 1wherein the processor-executable instructions or data of the computerprogram product which cause the robot to identify the first workflow tocomplete the first work objective cause the robot to: identify a firstpermutation of a first combination of reusable work primitives from thecatalog of libraries of reusable work primitives.
 11. The computerprogram product of claim 1 wherein the processor-executable instructionsor data of the computer program product further cause the robot toselect the first library of reusable work primitives from the catalog oflibraries of reusable work primitives based on the first work objective.12. The computer program product of claim 1 wherein theprocessor-executable instructions or data of the computer programproduct further cause the robot to select the first library of reusablework primitives from the catalog of libraries of reusable workprimitives based on the first workflow.
 13. The computer program productof claim 1 wherein the processor-executable instructions or data of thecomputer program product which cause the robot to identify the firstwork objective cause the robot to: identify the first work objectivebased on the first library of reusable work primitives.
 14. The computerprogram product of claim 1 wherein the processor-executable instructionsor data of the computer program product which cause the robot toidentify the first workflow cause the robot to: identify the firstworkflow based on the first library of reusable work primitives.
 15. Thecomputer program product of claim 1 wherein the catalog of libraries ofreusable work primitives includes a plurality of libraries of reusablework primitives directed to respective service categories, and whereinthe processor-executable instructions or data of the computer programproduct which cause the robot to identify the first work objective causethe robot to: identify the first work objective based at least in parton a service category; and select the first library of reusable workprimitives from the catalog of libraries of reusable work primitivesbased on a service category.
 16. The computer program product of claim15 wherein each library in the plurality of libraries is directed to arespective service category selected from a group of service categoriesconsisting of: medical service; cleaning service; housekeeping service;warehouse service; fetch service; delivery service; repair service;painting service; and retail service.
 17. The computer program productof claim 1 wherein the first library of reusable work primitives isselected from a plurality of libraries of reusable work primitives, eachlibrary of reusable work primitives including a plurality of workprimitives which is at least partially different from respectivepluralities of reusable work primitives included in other libraries ofreusable work primitives.
 18. The computer program product of claim 17wherein each library of reusable work primitives is defined according toa set of work objectives performable by reusable work primitivesincluded in the respective library of reusable work primitives.
 19. Thecomputer program product of claim 17 wherein each library of reusablework primitives is defined according to a place of use for therespective library of reusable work primitives.
 20. The computer programproduct of claim 1, the robot further including a communicationinterface communicatively coupled to the at least one processor, whereinthe processor-executable instructions or data of the computer programproduct which cause the robot to access the first library of reusablework primitives from the catalog of libraries of reusable workprimitives cause the robot to: access, via the communication interface,the first library of reusable work primitives stored on at least oneremote non-transitory processor-readable storage medium of a deviceremote from the robot.